#2022-04-1815:11pez😃 And now we have update-vals https://clojure.org/news/2022/03/22/clojure-1-11-0#2022-04-2119:17lepistaneHello,
I am trying to setup the https://github.com/hyperfiddle/rcf#ci using deps.edn for the project.
I've managed to do it (mostly) but after the tests execute terminal just 'hangs'. Is this normal? Doesn't feel normal...
this is my /test/app/core_test.clj
(ns app.core-test
(:require [clojure.test :as test]
[my.ns :as ns]))
(defn run-tests [& args]
(test/run-tests 'my.ns))
This is my deps edn test alias
{:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]
:extra-paths ["test"]
:exec-fn app.core-test/run-tests}
this is how i run it
clj -X:test
This is the output
and i had to ctrl + z to make it stop. (removed ns names but didn't go good job that's why you see some dots in the pic)
How do i prevent the 'hang' after it executes?
edit
Ok never mind... I just needed to wait bit longer ~60sec for it to exit. Is that ok?#2022-04-2120:07Dustin GetzThis is not normal, I just double checked that clj -M:test -e "(require 'user)(clojure.test/run-tests 'user)" returns instantly after running a hello world test#2022-04-2120:10Dustin GetzIf you send me a git repo i can clone it and take a look#2022-04-2204:44lepistaneI can't share the exact repo but i can make dummy repo to see if it works.
in my case
clj -M:test -e "(require 'app.core-test)(app.core-test/run-tests)"
has delay but
clj -M:test -e "(require 'app.ns.with.tests)(clojure.test/run-tests 'app.ns.with.tests)"
works instant.#2022-04-2205:03lepistanei made dummy repo and clj -X:test returns instantly#2022-04-2205:04lepistane(ns
(:require [hyperfiddle.rcf :refer [tests ! %]]))
(tests
"equality"
(inc 1) := 2
"wildcards"
{:a :b, :b [2 :b]} := {:a _, _ [2 _]}
"unification"
{:a :b, :b [2 :b]} := {:a ?b, ?b [2 ?b]}
"unification on reference types"
(def x (atom nil))
{:a x, :b x} := {:a ?x, :b ?x}
(tests
"nested tests (is there a strong use case?)"
1 := 1
"tests form returns final result"
(tests (inc 1) := 2 (inc *1)) := 3)
(tests
"REPL bindings work"
(keyword "a") := :a
(keyword "b") := :b
(keyword "c") := :c
*1 := :c
*2 := :b
*3 := :a
*1 := :c ; inspecting history does not affect history
(keyword "d") := :d
*1 := :d
*2 := :c
*3 := :b
(symbol *2) := 'c ; this does affect history
(symbol *2) := 'd))
(ns app.app-test
(:require [clojure.test :as test]
[]))
(defn run-tests [& args]
(test/run-tests '))
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.11.0"}
com.hyperfiddle/rcf {:mvn/version "20220405"}}
:aliases {:test {:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]
:extra-paths ["test"]
:exec-fn app.app-test/run-tests}}}
#2022-04-2205:23lepistaneis there a way to measure why it hangs?
I am thinking that original repo creates a lot of classes because there are a lot of of dependencies so it takes little bit to exit the process?
does this make sense?#2022-04-2205:56lepistaneOke yeah.
1 out of 3 namespaces is causing hang. I will filter it down further to see where the issue lies.
Thank you for being willing to look at this, it seems it's project/namespace specific.#2022-04-2119:22Nick McAvoyBy the way, this was amazing last night! It was a real privilege to have Dustin tell a roomful of us in Philadelphia all about it. https://www.meetup.com/Clojadelphia/events/284772608/#2022-04-2120:01jtthIs there a trick to getting an account on hyperfiddle? I think I signed up for an account four(?) years ago and haven’t heard anything#2022-04-2120:04Dustin Getzwe started over a bunch of times and never launched publicly. We're rolling out a Photon release to ~10 people starting next week - book an office hours on my public calendly link if you want to talk about it#2022-04-2120:05Dustin Getzthanks for asking#2022-04-2120:06jtthah! okay, i just might do that! thanks#2022-04-2120:08Dustin Getzdo it!#2022-05-1313:21Dustin GetzWould you like to be onboarded? It takes 2 hours on zoom to get through to the good stuff (client/server transfer)#2022-05-1313:22Dustin GetzAh but it is the Photon language that we are releasing, the CRUD UI stuff is not ready#2022-04-2418:23lepistanehow do i do https://github.com/hyperfiddle/rcf#ci with lein?
I can't figure it out#2022-04-2418:23Dustin Getzwe have never tried to use lein with rcf#2022-04-2418:24Dustin Getzcan you be more specific about what exactly you are trying to do#2022-04-2418:35lepistaneBasically just be able to use RCF with lein.
It would be great if i can 'lein test' and it works.
I'd be fine if i could set it up like i did with deps.edn few days ago where clj -X:test and it works.
Here is the setup i shared few days ago here for deps.edn
(ns app.app-test
(:require [clojure.test :as test]
[]))
(defn run-tests [& args]
(test/run-tests '))
deps.edn
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.11.0"}
com.hyperfiddle/rcf {:mvn/version "20220405"}}
:aliases {:test {:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]
:extra-paths ["test"]
:exec-fn app.app-test/run-tests}}}
#2022-04-2418:39lepistanemy main problem is i don't know how to replicate what exec-fn does in lein#2022-04-2418:40lepistanei found this https://github.com/technomancy/leiningen/blob/master/doc/PLUGINS.md#not-writing-a-plugin-%E6%97%A0%E4%B8%BA
but i am not sure it can help me#2022-04-2420:06lepistaneproject.clj
:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]
:test-paths ["test"]
:aliases {"mytest" ["run" "-m" "sis247.core-test/run-tests" :project/test-paths]}
in terminal
lein mytest
and it works 😄 😄 😄
I am so happy#2022-04-2420:06lepistanei will attempt to refine this little bit because i don't want global jvm-opts to be there#2022-04-2420:11lepistanealright
final solution that i am happy with.
Hope this helps someone in the future in case they are looking for how to run RCF using lein project.clj
project.clj
:test-paths ["test"]
:profiles {
:test {:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]
:aliases {"run-tests" ["run" "-m" "app.core-test/run-tests" :project/test-paths]}}
}
app_test.clj
(ns app.core-test
(:require [clojure.test :as test]
[app.config]))
(defn run-tests [args]
(test/run-tests 'app.config))
how to run it (terminal)
lein with-profile test run-tests
references:
https://github.com/technomancy/leiningen/blob/master/doc/PLUGINS.md#not-writing-a-plugin-%E6%97%A0%E4%B8%BA
https://github.com/technomancy/leiningen/blob/master/doc/PROFILES.md#2022-04-2420:19Dustin Getzsis247 is app ?#2022-04-2420:20Dustin GetzThank you for commenting the solution, I've added it to an issue for others to find#2022-04-2420:37lepistaneoh! that was a typo. I fixed it as well.
Good catch.#2022-04-2811:43Daniel JompheA potentially very nice minor convention with RCF to document our intentions when building small utils.#2022-04-2811:44Daniel JompheOf course we should write a docstring too.#2022-04-2811:45Daniel JompheSomething like this.
(defn join-not-blank
"Like clojure's str/join but does not separate str/blank? vals in coll.
`(join-not-blank \"🧱\" [\"a\" \"\" nil \"b\"]) := \"a🧱\"`"
([coll]
(str/join (remove str/blank? coll)))
([separator coll]
(str/join separator (remove str/blank? coll))))
(rcf/tests
;; What we didn't like about str/join...
(str/join "🧱" ["a" "" nil "b"]) := "a🧱🧱🧱"
;; So we built this...
(join-not-blank "🧱" ["a" "" nil "b"]) := "a🧱b"
(join-not-blank ["a" "" nil "b"]) := "ab"
(join-not-blank "🧱" []) := ""
(join-not-blank []) := ""
;;
)
#2022-04-2812:11Daniel JompheGotta make time to create an utility to auto-append RCF tests to the end of function docstrings!#2022-04-2812:20Dustin GetzYeah it would be great to integrate with docstrings etc; it also might be possible to get editors to offer an interface for examples, RCF could set :tests meta on vars#2022-04-2812:25Daniel JompheI will explore solutions external to RCF. If I reach some learnings about IDE UIs that might be useful to RCF too, I might ping you in e.g. #calva where I'll show some things to these folks.#2022-04-2909:59Ben SlessCould be worth looking at now CIDER for example can display specs alongside docstrings#2022-05-0320:52Daniel JompheGeoffrey is already up to date on this but here's the entire code of my current prototype. I'm stopping improving it for a number of weeks (focusing on something else), but we're already satisfied with the following...
With this, RCF tests can now (still) be auto-executed, but also auto-added to VS Code Calva's doc hover tooltips.
https://clojurians.slack.com/archives/CBE668G4R/p1651609025359479?thread_ts=1651259227.960969&cid=CBE668G4R#2022-04-2812:11Daniel JompheGotta make time to create an utility to auto-append RCF tests to the end of function docstrings!#2022-05-0213:28Daniel Basneris the demo you linked to on twitter at the end of the hytradboi talk (great job btw!) different from previous demos? I have twitter blocked on my devices so I cannot check haha#2022-05-0213:35Dustin GetzThe CRUD-app-as-an-expression hypermedia demo did not make it out on time; soon! I haven't had time to narrate and produce a good video#2022-05-0213:35Dustin Getzprobably next week#2022-05-0213:36Daniel Basnerok, take your time! If you could post it here in addition to twitter I’d be very appreciative#2022-05-0214:17Dustin GetzSure, will do#2022-05-0214:19Dustin GetzFor those out of the loop, we launched a Photon 10min lightning talk on Friday which became public on Saturday.
Video link and hype:
https://twitter.com/dustingetz/status/1520397540386091009
https://news.ycombinator.com/item?id=31217448
Best place for Q&A is r/clojure:
https://www.reddit.com/r/Clojure/comments/ufbzp6/uis_are_streaming_dags_hyperfiddlephoton_a/#2022-05-0214:24nottmeyHi Dustin, I really like the Idea and would love to see this architecture compete against nuxt.js, next.js, etc. frameworks.
How can outside people help you progress with that? 🙂
(Besides boosting the tweet 😄)#2022-05-0214:31Dustin GetzIf you have some free time, you can try it out with me on zoom (I will help you clone the repo and manually onboard you), interested? First three people are trying it in the next week or so#2022-05-0214:42Daniel Jomphehttps://www.reddit.com/r/programming/comments/ufskmc/uis_are_streaming_dags_video/.#2022-05-0217:55Ben SlessSince this is equivalent in some sense to inlining, how do you solve the problem of closures duplicating work?#2022-05-0218:06Dustin GetzClosures are higher order DAGs (DAG as a Value); this has monadic structure and therefore the DAG is dynamic, imagine a DAG evolving/animating over time as you navigate through screens of a UI the DAG updates to reflect the new set of edges that are active. This structure is called "self adjusting computation" per https://blog.janestreet.com/breaking-down-frp/#2022-05-0218:07Dustin GetzThe article is very old (2014), this statement from the article is flat out wrong: "As you might expect, the lack of history-sensitivity makes SAC less suitable for writing user interfaces"#2022-05-0218:07Ben Slesswelp, time to do some reading#2022-05-0218:08Ben SlessI know HVM managed to solve graph reduction from one direction by implementing copying correctly, now I got to understand what you did#2022-05-0218:08Dustin GetzA simpler hello world case to reason about first is an if statement, the compiler knows statically that there are two branches and only one can be active at a time and the DAG can switch between them. Even this simple DAG has a degree of dynamism, imagine the DAG animating as it flips between the two states#2022-05-0218:25Ben Slesswhen about a case like:
(fn [x] (let [x (+ n 2)] (fn [y] (+ y x)))
This function returns a graph, will x's expression be inlined in it?#2022-05-0218:26Ben SlessI see how an if doesn't duplicate work in call-by-need, but not sure regarding the example#2022-05-0218:51Dustin Getz“DAG” “AST” “lisp form as data structure” are all essentially synonyms, all are abstract until an evaluator is provided. So it is 100% okay to reason about this example as regular lisp closures; the binding named by x is what is closed over. it turns out that if you provide a reactive eval you will get a reactive closure that does “the right thing”. The term “DAG” here can just be thrown away, it's just a helpful metaphor#2022-05-0218:56Dustin Getzif you want to reason about it as a DAG: let creates a named edge, closures are DAG values, and you'd need to recognize DAG-of-DAG structure as symmetrical to promise-of-promise; the evaluator flattens it at runtime#2022-05-0218:59Dustin Getzsame structure as fn of fn; you need to actually evaluate and run the fns to flatten the structure and see which way the fns went per whatever runtime control flow logic around them#2022-05-0219:08Ben Slessmy question essentially is what are the call semantics, call by value or call by need#2022-05-0219:08Ben SlessLet's take this example:
(defn foo [n] (let [x (+ n 2)] (fn [y] (+ y x))))
(defn bar
[]
(let [f (foo 5)]
(map foo [1 2 3])))#2022-05-0219:09Ben SlessIf we do evaluation by substitution then we might end up calculating (+ n 2) 3 times#2022-05-0219:10Ben SlessIf evaluate by value then it's evaluated once#2022-05-0219:29Ben Slesssee section 2.3 https://www.microsoft.com/en-us/research/wp-content/uploads/2002/07/inline.pdf#2022-05-0223:26Dustin Getz> Since this is equivalent in some sense to inlining
> A good compiler must inline many of these calls to recover an eciently executable program
I dont think what we're doing is inlining#2022-05-0223:40Dustin GetzThis passes, (+ n 2) is computed once#2022-05-0223:45Dustin Getzp/fn are capitalized and called with (new) aka . for great reasons:
1. for full clojure compat we need to support regular both normal Clojure function calls (trace!, range, m/watch, rcf/!) and reactive function calls (Bar, Foo, F)
2. without a type inferencer, you need syntax to disambiguate the two types of calls (same as hiccup). The compiler could infer top level static calls with metadata on var, but the compiler cannot infer dynamic calls, so you need syntax
3. It turns out that reactive functions can be seen as actors, Photon functions are objects, they have a history of states over time, they have a component lifecycle like React, you send events/messages to them -> they maintain their state -> emit new events/messages. So they are called/booted/constructed with (new)
4. foreign missionary flows are also joined with (new) - as in (new (m/watch ...)), it's not a coincidence#2022-05-0300:09Dustin GetzHere is the compiler artifacts, (steady x) is (m/watch (atom x)); (latest-apply f a b) is (m/latest f a b) with some error handling. This is as deep as I go, beyond this point is "Leo stuff"#2022-05-0300:11Dustin Getzthe key point is that p/fn ends up becoming a real clojure fn; let is real clojure let, it's the bindings that are all lifted into missionary signals, so the compiler basically calls Photon functions via m/latest#2022-05-0312:34Ben SlessThat's a lot to take in#2022-05-0313:06Dustin GetzThank you for saying that so I have the feedback - this is my first time trying to explain it#2022-05-0313:06Dustin GetzThe answer to your question is it's "call by value"#2022-05-0315:34DameonI’m interested in trying to work with this. The model seems super compelling.#2022-05-0221:37Daniel JompheFor hyperfiddle's rcf/tests macro, let me suggest you normalize and suggest every user to require it aliased instead of referred, so that everywhere on the web, where someone posts screenshots like the following one, people searching google might find your lib easily.#2022-05-0519:21Peter NagyHi everyone! My name is Peter and I work at a clojure startup called Fluent. We don't have a public alpha (or any alpha) yet, so not much to show currently, sorry! Based on https://old.reddit.com/r/Clojure/comments/ufbzp6/uis_are_streaming_dags_hyperfiddlephoton_a/i74ve5e/ I'm acquiring if there is any space left for a clojure nerd like me to enter your private alpha testing of Photon?
Some background about me: I learned clojure in its very early version, but didn't know any Java so I left it for a couple of years. I dabbled with many languages since then and found my way back to clojure (there really isn't anything better out there, I looked!). I spent 1-2 years of my career writing Common Lisp and it's about 4 years of Clojure career at this point as well. https://github.com/xificurC . I closely followed Leo's work in Missionary and contributed the RxJava comparison tutorial to the repo.
Why I'd like to join the private alpha: currently we're using re-frame for building our platform, which works OK for most cases. But Photon looks like a promising alternative that could really simplify the codebase in the long run! I'd be happy to check it out and report back to the team and see if there's any chance of adopting it.
To be fully transparent I'll add that I don't have frontend experience apart from the bits I worked with it here at Fluent, where I'm working since January. I don't think it should be too big of a stumbling block though, I'm a quick study! 🙂#2022-05-0520:43Dustin GetzGreat, you're on the list, I will DM you to schedule a zoom call to onboard you (as we don't have docs yet)#2022-05-0607:30Peter Nagygreat! looking forward to it#2022-05-0608:47henrikHi! My name is Henrik.
I'm with the same company as Peter, so my motivations are largely the same as Peter. Having spent a lot of time and energy on the API layer recently, I'm super stoked about the idea of Photon.
In contrast to Peter, I do have a fair amount of frontend experience. :) My background is that I worked as a UX/service designer for about ten years. My initial interest in Clojure stemmed from watching Simple Made Easy and going, "well, this elegantly expresses a lot of the problems I encounter in design, but in another domain". I started learning Clojure as a way to educate myself to be a better designer, and soon started making prototypes in ClojureScript and Reagent, rather than the WYSIWYG tools available at the time. Skip ahead to the current day via a slippery slope of increasing interest and responsibilities within development, and I'm in charge of the architectural design at Fluent.
I've kept my eyes on Photon since you initially started talking about it on Twitter. Via Peter, I've also been introduced to Missionary, and have started using it quite a bit.
I'd love to join the call.#2022-05-0612:21Dustin GetzSounds great! Thank you!#2022-05-1120:40Dustin Getz#2022-05-1122:10kennytiltonNice. Saw the Diamond test. Do you handle what Philip Eby christened "The Pentagram of Death"? I cannot find his description on The Google, but it was the use case that convinced him my Cells algorithm was the only way to ensure data integrity. The key elements are:
• two dependency chains from some property P leading back to one property X; and
• branching code in the derivation of P that will not travel the second dependency chain until a certain condition is met; and
• by chance, propagation reaches P on the the first path before it reaches some intermediate property I on the second dependency chain.
The consequence is P updating once and reading (for the first time) property I, which has not yet been updated hence is inconsistent with the new value of X. This inconsistency is temporary (hence the name "glitch") because I will be updated soon enough and P will achieve consistency with X, but if one's reactive engine dispatches side effects off state change -- possible trouble.#2022-05-1122:14Dustin GetzThis is a missionary layer question, but regardless show me a concrete test and i'll port it to photon#2022-05-1122:18kennytiltonThe Google fails me on "missionary layer question", but I think I have the test in my Cells regression test. Guessing Clojure will be OK? 🙂#2022-05-1122:19Dustin Getzsure!#2022-05-1122:19Dustin Getzhttps://github.com/leonoel/missionary , Photon's DAG compile target (which has a design goal of being glitch free)#2022-05-1122:45kennytiltonOhhh, understood on "missionary layer". I forgot about that component. I am afraid my regression test will not be of much use -- it is quite unreadable, and was written by someone who was just thinking about how to trigger the glitch (me) -- and may not even translate to a streaming model: https://github.com/kennytilton/matrix/blob/73d58c985fcc87ca2d817b4c9215643acbeb4c45/cljs/matrix/test/tiltontec/cell/evaluate_test.cljc#L147
"may not even translate"? I saw Andrew Staltz conceding ReactiveX was susceptible to glitches: https://staltz.com/rx-glitches-arent-actually-a-problem.html The "solution" of coding around them was unsatisfactory, btw -- glitches arise in deep computations we really do not want to expand into the glitchy computation.
MobX, btw, happens to do what Cells does: when reading a property during a computation, check that it is current and if not make it so before answering the read! JIT integrity. But that idea is what I think may conflict with a system accepting a stream of values.
I wager the Missionary folks have thought this thru: a single datum arrives in a reactive system and like a particle/wave passes thru multiple slits. Does missionary guarantee that any and all dependencies experience only one new arrival of data?#2022-05-1122:49Dustin Getz> Does missionary guarantee that any and all dependencies experience only one new arrival of data?
Yes, glitches are unacceptable in our opinion#2022-05-1122:50Dustin GetzAs you pointed out, events often drive external side effects, so there cannot be glitches#2022-05-1123:08Dustin GetzIs this close to the test?
(tests
(def !p (atom 0))
(def !q (atom 1))
(def !control (atom :p))
(with (p/run (! (let [p (p/watch !p)
q (p/watch !q)
control (p/watch !control)
cross (case control :p p :q q)]
[p q control cross])))
% := [0 1 :p 0]
(reset! !control :q)
% := [0 1 :q 1]))#2022-05-1123:13Dustin GetzI think it's the same test as what you describe; it's the self-adjusting case where cross changes from p to q, and q has never been seen#2022-05-1123:17Dustin GetzHere it is using your letters, I think
(tests
(def !x (atom 0))
(def !control (atom :a))
(with (p/run (! (let [x (p/watch !x)
control (p/watch !control)
y (inc x)
p (case control :a x :b y)]
p)))
% := 0
(reset! !control :b)
% := 1))
#2022-05-1208:21kennytiltonHmm, good idea! Let me fire up Missionary and get to know it for its own sake. I was able to demonstrate glitches with Reagent, I will see what I can do with Missionary.#2022-05-1208:30kennytiltonOh, wait, they say "Correct incremental maintenance of dynamic DAGs without inconsistent states (aka FRP glitches)". There ya go!#2022-05-1208:52kennytiltonChecking anyway... 🙂#2022-05-1208:59leonoelthis is the pentagram of death in missionary
(def !aa (atom 1))
=> #'user/!aa
(def !a7 (atom 7))
=> #'user/!a7
(def cancel
((m/reactor
(let [<aa (m/signal! (m/watch !aa))
<a7 (m/signal! (m/watch !a7))
<a70 (m/signal! (m/latest (partial * 10) <a7))
<bb (m/signal! <aa)
<cc (m/signal! (m/latest (partial * 10) <aa))
<dd (m/signal! (m/cp
(try
(if (even? (m/?< <bb))
(* 10 (m/?< <cc)) 42)
(catch Cancelled _))))
<ee (m/signal! (m/latest + <a70 <bb (m/latest (partial * 10000) <dd)))]
(m/stream!
(m/ap
(m/amb=
(prn 'aa (m/?< <aa))
(prn 'a7 (m/?< <a7))
(prn 'a70 (m/?< <a70))
(prn 'bb (m/?< <bb))
(prn 'cc (m/?< <cc))
(prn 'dd (m/?< <dd))
(prn 'ee (m/?< <ee)))))))
prn prn))
ee 420071
dd 42
cc 10
bb 1
a70 70
a7 7
aa 1
=> #'user/cancel
(swap! !aa inc)
aa 2
bb 2
cc 20
dd 200
ee 2000072
=> 2
(swap! !aa inc)
aa 3
bb 3
cc 30
dd 42
ee 420073
=> 3
It is pretty much what the photon version would desugar to :
(p/run
(let [aa (p/Watch. !aa)
a7 (p/Watch. !a7)
a70 (* 10 a7)
bb aa
cc (* 10 aa)
dd (if (even? bb)
(* 10 cc)
42)
ee (+ a70 bb (* 10000 dd))]
(prn 'aa aa)
(prn 'a7 a7)
(prn 'a70 a70)
(prn 'bb bb)
(prn 'cc cc)
(prn 'dd dd)
(prn 'ee ee)))
function calls are turned into m/latest, let bindings to m/signal! etc#2022-05-1209:01leonoelif you want to try it at home, build latest missionary locally, cp is not released yet#2022-05-1209:13kennytiltonDamn, you all are fast! What does "build locally" look like, @U053XQP4S? All I know is lein install. Thx for this!#2022-05-1209:14leonoelmvn clean && mvn compile && mvn install#2022-05-1209:15leonoelthen use maven version b.27-SNAPSHOT#2022-05-1210:37Dustin GetzKenny I'll onboard you to Photon if you want to try it out, we're in private tech alpha, there are several good tutorials and growing fast, including some missionary tutorials #2022-05-1213:19PanelPhoton looks like a wrapper on top of missionary with nicer syntax, is that right ?#2022-05-1213:22kennytiltonThx, @U053XQP4S, install went smoothly. I have worked through the two tutorials, will check out your Pentagram shortly.#2022-05-1213:25kennytiltonThx, @U09K620SG. Is there a sign-up thingy you use? Right now I am wrapping #clojuredart in Matrix, but I would like to keep up with Photon. I have visions of a specialty consulting gig.#2022-05-1213:44kennytiltonGetting close, @U053XQP4S :
Starting nREPL server...
/Users/kennethtilton/jdks/zulu11.35.15-ca-fx-jdk11.0.5-macosx_x64/bin/java -Dfile.encoding=UTF-8 -XX:-OmitStackTraceInFastThrow -Dclojure.compile.path=/Users/kennethtilton/dev/words/target/default/classes -Dwords.version=0.1.0-SNAPSHOT -Dclojure.debug=false "-javaagent:/Applications/IntelliJ IDEA IDEA " -classpath /Users/kennethtilton/dev/words/test:/Users/kennethtilton/dev/words/src:/Users/kennethtilton/dev/words/dev-resources:/Users/kennethtilton/dev/words/resources:/Users/kennethtilton/dev/words/target/default/classes:/Users/kennethtilton/.m2/repository/nrepl/nrepl/0.6.0/nrepl-0.6.0.jar:/Users/kennethtilton/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.8.7/jackson-core-2.8.7.jar:/Users/kennethtilton/.m2/repository/org/clojure/clojurescript/1.10.879/clojurescript-1.10.879.jar:/Users/kennethtilton/.m2/repository/clojure-complete/clojure-complete/0.2.5/clojure-complete-0.2.5.jar:/Users/kennethtilton/.m2/repository/org/msgpack/msgpack/0.6.12/msgpack-0.6.12.jar:/Users/kennethtilton/.m2/repository/org/clojure/tools.reader/1.3.3/tools.reader-1.3.3.jar:/Users/kennethtilton/.m2/repository/org/clojure/tools.analyzer/0.6.9/tools.analyzer-0.6.9.jar:/Users/kennethtilton/.m2/repository/org/clojure/google-closure-library/0.0-20201211-3e6c510d/google-closure-library-0.0-20201211-3e6c510d.jar:/Users/kennethtilton/.m2/repository/org/clojure/core.memoize/0.5.9/core.memoize-0.5.9.jar:/Users/kennethtilton/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20201211-3e6c510d/google-closure-library-third-party-0.0-20201211-3e6c510d.jar:/Users/kennethtilton/.m2/repository/missionary/missionary/b.27-SNAPSHOT/missionary-b.27-SNAPSHOT.jar:/Users/kennethtilton/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20210505/closure-compiler-unshaded-v20210505.jar:/Users/kennethtilton/.m2/repository/com/googlecode/json-simple/json-simple/1.1.1/json-simple-1.1.1.jar:/Users/kennethtilton/.m2/repository/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar:/Users/kennethtilton/.m2/repository/cloroutine/cloroutine/10/cloroutine-10.jar:/Users/kennethtilton/.m2/repository/org/ow2/asm/asm-all/4.2/asm-all-4.2.jar:/Users/kennethtilton/.m2/repository/org/clojure/data.json/0.2.6/data.json-0.2.6.jar:/Users/kennethtilton/.m2/repository/com/cognitect/transit-java/0.8.332/transit-java-0.8.332.jar:/Users/kennethtilton/.m2/repository/org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.jar:/Users/kennethtilton/.m2/repository/com/cognitect/transit-clj/0.8.309/transit-clj-0.8.309.jar:/Users/kennethtilton/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/kennethtilton/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/kennethtilton/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar:/Users/kennethtilton/.m2/repository/org/clojure/core.cache/0.6.5/core.cache-0.6.5.jar:/Users/kennethtilton/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/kennethtilton/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/kennethtilton/.m2/repository/org/clojure/tools.analyzer.jvm/0.7.2/tools.analyzer.jvm-0.7.2.jar clojure.main -i /private/var/folders/zl/tyjfph_92ws1z65_7mbtbdxc0000gn/T/form-init8951147573957717648.clj
:user-loaded!!!!!!!!!!!!!!!
Connecting to local nREPL server...
Clojure 1.10.1
nREPL server started on port 49534 on host 127.0.0.1 -
Loading test/missionary/glitch_test.clj... done
(def !aa (atom 1))
=> #'missionary.glitch-test/!aa
(def !a7 (atom 7))
=> #'missionary.glitch-test/!a7
(def cancel
((m/reactor
(let [<aa (m/signal! (m/watch !aa))
<a7 (m/signal! (m/watch !a7))
<a70 (m/signal! (m/latest (partial * 10) <a7))
<bb (m/signal! <aa)
<cc (m/signal! (m/latest (partial * 10) <aa))
<dd (m/signal! (m/cp
(try
(if (even? (m/?< <bb))
(* 10 (m/?< <cc)) 42)
(catch Cancelled _))))
<ee (m/signal! (m/latest + <a70 <bb (m/latest (partial * 10000) <dd)))]
(m/stream!
(m/ap
(m/amb=
(prn 'aa (m/?< <aa))
(prn 'a7 (m/?< <a7))
(prn 'a70 (m/?< <a70))
(prn 'bb (m/?< <bb))
(prn 'cc (m/?< <cc))
(prn 'dd (m/?< <dd))
(prn 'ee (m/?< <ee)))))))
prn prn))
Syntax error macroexpanding cloroutine.core/cr at (test/missionary/glitch_test.clj:8:29).
Could not resolve var: Cancelled
My NS:
(ns missionary.glitch-test
(:require [clojure.test :refer :all]
[missionary.core :refer [?] :as m]))
My leiningen project.clj:
(defproject words "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url ""
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url ""}
:dependencies [[org.clojure/clojure "1.10.1"]
[missionary "b.27-SNAPSHOT"]]
:main ^:skip-aot words.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all
:jvm-opts ["-Dclojure.compiler.direct-linking=true"]}})
The relevant build output:
-----------------------< missionary:missionary >------------------------
[INFO] Building missionary b.27-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ missionary ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 17 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ missionary ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ missionary ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/kennethtilton/dev/missionary/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ missionary ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ missionary ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ missionary ---
[INFO] Building jar: /Users/kennethtilton/dev/missionary/target/missionary-b.27-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ missionary ---
[INFO] Installing /Users/kennethtilton/dev/missionary/target/missionary-b.27-SNAPSHOT.jar to /Users/kennethtilton/.m2/repository/missionary/missionary/b.27-SNAPSHOT/missionary-b.27-SNAPSHOT.jar
[INFO] Installing /Users/kennethtilton/dev/missionary/pom.xml to /Users/kennethtilton/.m2/repository/missionary/missionary/b.27-SNAPSHOT/missionary-b.27-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.507 s
[INFO] Finished at: 2022-05-12T09:21:02-04:00
[INFO] ------------------------------------------------------------------------
Any suggestions? Thx for the extra effort, btw!#2022-05-1213:48kennytiltonbtw, I am running Intellij/Cursive evaluating all expressions via the repl.#2022-05-1213:52leonoel(import missionary.Cancelled)#2022-05-1214:13Panelopen-sourcing photon is part of the plan ?#2022-05-1214:27Dustin GetzYes of course, our business model is cloud low-code stuff#2022-05-1214:29Dustin GetzIt is only private today because we want to iron out the rough edges and make sure initial trials are successful. Been onboarding people this week, so far so good!#2022-05-1120:45Noah Bogartthat looks great#2022-05-1214:46Dustin Getz(tests
"pentagram of death - via Kenny Tilton"
(def !aa (atom 1))
(def !a7 (atom 7))
(with
(p/run
(let [aa (p/watch !aa)
a7 (p/watch !a7)
a70 (* 10 a7)
bb aa
cc (* 10 aa)
dd (if (even? bb)
(* 10 cc)
42)]
(! (+ a70 bb (* 10000 dd)))))
% := 420071
(swap! !aa inc)
% := 2000072
(swap! !aa inc)
% := 420073))
✅✅✅=> 420073#2022-05-1214:49Dustin Getz#2022-05-1214:49Dustin Getz✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅=> {ee 420073}#2022-05-1214:51Dustin GetzYou will need RCF master branch to run these (not the published maven coords)#2022-05-1214:53Dustin GetzI think this one is the most reduced form that matches the text description (edited)
(tests
"pentagram of death reduced"
; the essence of the problem is:
; 1. if/case switch/change the DAG (imagine a railroad switch between two train tracks)
; 2. to have a conditional where the predicate and the consequent have a common dependency
(def !x (atom 1))
(with (p/run (! (let [p (p/watch !x)
q (! (str p))
control (- p)]
(case control -1 p -2 q q))))
% := 1 ; cross
; q has not been touched, no causal dependency yet
(swap! !x inc)
% := "2" ; q first touched
% := "2"))
✅✅✅#2022-05-1214:56leonoelas I understand it the essence of the problem is to have a conditional where the predicate and the consequent have a common dependency#2022-05-1214:59Dustin GetzOops you're right, editing#2022-05-1214:59Dustin GetzIs it correct now?#2022-05-1215:01leonoelcontrol should depend on p#2022-05-1215:21Dustin Getz#2022-05-1215:37PanelWhy p/for on 50 and not jut core for ?#2022-05-1215:42PanelDo you have another layer on top of photon to mange dom updates ?#2022-05-1216:18Dustin Getzp/for is concurrent and does diffing to stabilize the body over time. There is an object lifecycle (mount/unmount). See the final test in photon-1-hello-world, line 127#2022-05-1216:19Dustin GetzYes, we have a photon-dom library not shown yet in these tutorials#2022-05-1215:52kennytiltonStill getting up to speed on your code, but the key test is whether a derived property takes on an intermediate inconsistent value before finally converging on the correct value consistent with the single new input.
This then cannot be ascertained by looking at final values.
You are right, however, that the key to triggering the problem is a condiitonal branching to a Cell never before consulted, the backstory here is defeating any suggestion that a dynamic examination of DAG could avoid the glitch.
If we look at my original example, this explains the code that worries about how many times a computation of the downstream-most cell was done after a single increment to the primary source input cell.
If only one computation was done, and we have the correct answer, all is well.
Come to think of it, I might have tightened the test by trapping each computation and confirming (a) there was only one and (b) if there were two, that one of them was inconsistent with the new input. I was probably distracted from doing so because the Cells contract also guarantees that formulas and observers run only once. But as long as we got only one computation, and the right one, correctness was confirmed.
I got sidetracked by a plumber, but I have the code working now and I will try to see if I can confirm. Oops, plumber is back.#2022-05-1216:14kennytiltonAlso, still studying, am I right that the test just gets as far as the initial computations? The crux comes when we increment the origin aa to 2 (even); bb copies it, and dd for the first time consults cc. The problem is if propagation of aa being now 2 reaches cc after dd has run and consumed the prior value of cc, thus coming up with a result inconsistent with aa=2.
fwiw, the way both MobX and Cells work is to detect (by different mechanisms!) that cc is a change generation too old, and recomputes cc before responding to the read. Cells uses a serially increasing "pulse", and tbh I forget what MobX does to determine obsolescence.#2022-05-1216:15kennytiltonI meant to ask how to increment aa. 🙂#2022-05-1216:18kennytiltonAh, vanilla swap!. 🙂 Looks promising -- I had added a print to the dd computation and it ran only once.#2022-05-1216:22Dustin Getzrcf/! is basically println, it is the "test probe" that captures when it was called so that the order of "test probe effects" can be asserted with %#2022-05-1216:39kennytiltonSo all looks good. The only question is whether a chance internal ordering is making the test pass, but I move the definitions around and nothing broke the test. Nice work, @leonoel!#2022-05-1514:50kennytiltonOk, now that I know a bit more thx to @U09K620SG’s excellent onboarding examples, I have created a sample issue:
(tests "duplicate watches via fn's"
(def !x (atom 0))
(p/defn comp-y []
(! (+ 40 (p/watch !x))))
(p/defn comp-z []
(* 2 (p/watch !x)))
(def dispose
(p/run
(let [z (comp-z.)
y (comp-y.)]
(! (+ y z)))))
% := 40
(swap! !x inc)
% := 42
% := 44
(dispose))
This then is committing the documented sin of watching !x twice, but can this always be avoided in real-world code? The use case that broke Cells originally was a RoboCup Simulation involving a DAG where all new data arrived every 100ms at a single variable. So it was a great way to create glitches, and in my case part of the code did strategic thinking while another part tried to execute an immediate task, such as "kick ball". What happened was that the strategist changed course and the task being executed suddenly ceased to exist (but some code was trying to work with it).
Had Cells not had a glitch, the task would have been quiesced in an orderly fashion as part of the change in strategy, and the code would not have attempted to interact with that task.
The big point here is that it would be hard to avoid coding up duplicate watches as the code base grows and we cannot easily "see" the dependencies derived by calling heavy-lifter functions. And if I need to call both, I cannot avoid the duplication without combining the two functions in some monolith that will consult !x only once. That was Staltz's unattractive proposal.
In some cases, the system transiently taking on the value 42 could be a problem, but I gather from Andrew Staltz's protestations that the RX stream crowd just lives with glitches. Perhaps there coding practices that make them mangeable, and to be honest, I wrote a lot of quite hairy enterprise code with Cells before RoboCup broke it, so HF should be fine as is.
The solution that Gelerntner envisioned, and that the next version of Cells implemented, was for the DAG to move forward as one after any change. Like Conway's Game of Life: all cells decide their next state looking at all cells in the current state, and all change together. Cells and MobX work kinda the opposite: when they recompute after a change, they are guaranteed they will see only new state, consistent with the original state change.
In stream-speak, I believe Mike of re-frame talks about "de-duping" the signal graph.#2022-05-1514:51Dustin Getzp/watch is a macro to hide something from tutorial usage, look at the impl#2022-05-1514:53Dustin GetzPull latest to see this new tutorial#2022-05-1515:16kennytiltonOh, I thought watch was how we read a value and established the reactive dependency, such that changes would flow thru automatically. Not so?#2022-05-1515:17kennytiltonOnly diff I see in the new example is using new instead of dot notation. Did I miss sth else?#2022-05-1515:19Dustin GetzThe point of the example is if you only want one subscription to the atom, call new once#2022-05-1515:20Dustin Getzm/watch does not actually perform side effects, it returns a pure functional "IO value" that describes a side effect#2022-05-1515:22Dustin Getzbetter explanation - m/watch returns a flow, but it does not run the flow. You have to instantiate it#2022-05-1515:24Dustin Getzp/watch is different then m/watch btw - did you see that they are different? (p/watch !x) macroexpands to (new (m/watch !x)), bundling the instantiation (new) with the flow definition (m/watch)#2022-05-1515:24Dustin Getzp/watch maybe should have a ! as in p/watch!#2022-05-2715:56Daniel JompheSo nice to have Kenny and Dustin in the same room. 😄 I was reading you quite intently about Cells in e.g. 2006, Kenny, and a few years ago when Matrix surfaced too.
What was your conclusion about this discussion, did everything check up as being glitch-free?#2022-05-2804:43kennytiltonNo, @U0514DPR7, if a derivation D pulls from the same source data point S via more than one path, a single change to S can lead to two or more computations of D, possibly with earlier computations then being transiently inconsistent with the change to S. Aka, a "glitch". These inconsistencies are rectified by the time propagation of the original change is complete.
These glitches are accepted by the reactive streams crowd as far as I can make out. They propose avoiding glitches by not coding multiple possible dependency paths back to a given source. ie, Glitches are "pilot error".
Cells had the same glitch until 3.0, and I was able to write a lot of quality code despite the theoretical flaw. Luck ran out with an extreme case of duplicate dependency paths, where all derivations led back to a single source. RoboCup simulation, in fact.#2022-05-2810:46Dustin Getzkenny we would appreciate if you would speak with less certainty wrt missionary’s design. duplicate watch is not the same thing as a glitch. missionary, in addition to being a glitch-free streaming library, is also a pure functional programming library with referentially transparent primitives (like Haskell). registering callbacks on an atom with watch is a side effect and pure FP requires us to treat that with a certain reverence and that is the essence of why missionary works differently here than you want it to. the behavior you observed is not caused by the propagation engine but rather the clojure IWatchable interop.#2022-05-2810:58Dustin Getzwe could change the behavior trivially if we wanted to but we feel it is a violation of FP and unvigorous#2022-05-2821:03kennytiltonI will be honest, @U09K620SG, I did not know what to make of your response to my demonstration of a glitch in the code above, so I let it go. Also, you had already described one glitchy example as a user error, so I thought you like Stalz were OK with the double updates (with one logically inconsistent update). But here we are. Was there something wrong with my example? It does update twice, and at one point comes up with a value logically inconsistent with the application state. If that is nominal behavior, fine, our definitions differ.#2022-05-2821:22Dustin Getz> Was there something wrong with my example?
yes#2022-05-2821:22Dustin Getz> If that is nominal behavior, fine, our definitions differ.
the busted program you provided is trivially fixed (subscribe to the atom once, not twice) and the issue has nothing to do with the propogation engine; it is a clojure interop issue#2022-05-2821:23Dustin GetzI have provided passing tests documenting all behaviors in the repo with many comments#2022-05-2821:34Dustin Getz> I thought you like Stalz were OK with the double updates (with one logically inconsistent update)
I want to be crystal clear to all in the future who will stumble upon this thread
1. we are NOT okay with double updates
2. the updates are NOT logically inconsistent, missionary has done exactly the correct thing because the program explicitly subscribes to the foreign clojure atom twice, and missionary correctly propagates immediately after the first foreign callback firing from the atom, and again after the second foreign callback firing from the atom. the programmer did not express the intended DAG, and we have great deeply thought out reasons for it being this way which are essential properties for client/server distribution#2022-05-2911:25kennytilton"the busted program you provided is trivially fixed (subscribe to the atom once, not twice) "
I tried to address this point ^^^ by pointing out that the dup subscribe to S could arise, not thru deliberately subscribing twice, but because the developer in a real app trying to derive D may well end up calling an interesting function IFN that happens also to refer to S along the way. But D still needs the raw value of S, which IFN obscures producing its own interesting result.
Cells 2.0 was broken by RoboCup's big strategic module tripping up the big skill execution module.
Staltz suggested copying all of IFN's logic into the derivation of D, or at least refactoring so dup subs do not arise. So I would have to roll the strategic and execution code into one blob, because of the reactive engine I chose? And would it always be trivial?
I am not saying FB engineers are any good, but there are a lot of them and Redux got invented because they simply could not figure out how to display accurately an unread message count: https://www.youtube.com/watch?v=nYkdrAPrdcw&t=783s As you listen to the speaker's breakdown of the challenges they faced with that simple message count, one can almost hear the glitches in their manual orchestration attempts. They would fix it and refix it constantly. That is precisely why Redux got invented, because an ad hoc solution eluded them. So telling users to just reorder their logic may not scale.
I was asked here what I had concluded about Missionary being susceptible to glitches, so I answered that it is, where a glitch is defined as a reactive value transiently taking on an inconsistent state visible to application code. We can call that a feature of Missionary instead of a glitch since it has been documented, but the window of risk is the same: complex applications will unavoidably lead to duplicate subscriptions and transient incorrect program state during state change propagation.#2022-05-2911:48Dustin GetzKenny i'm sorry you're extremely confused, the issue is at the clojure IWatchable interop layer, not the propogation engine#2022-05-2911:50Dustin GetzIf you are interested in understanding, the documentation is here: https://github.com/hyperfiddle/photon/blob/567860954be3db440ff663309ad2f54f301bdff5/src-docs/user/photon_missionary_interop.clj#L20-L41#2022-05-2912:32kennytilton"Kenny i'm sorry you're extremely confused,..."
It is the long COVID, I swear...
"..., the issue is at the clojure IWatchable interop layer, not the propogation engine"
So when my app screws up a $50b trade, my users sue...IWatchable? To a reactive engineer, things like IWatchable are our problem to work around, and no excuse for the behavior of the code I produce.
If we want to write serious reactive code, we have to own the responsibility for it. If perchance IWatchable was a great solution, sure, we use it. But if using it leads to glitches....
The response I have heard consistently is "hey, that is not our behavior, that is the behavior of this random tool we pulled off the shelf." How does that help my users?
I chose the tool, I have to make it work correctly.#2022-05-2912:40Dustin GetzUsing IWatchable does not lead to glitches. Subscribe to each atom once and you will have the intended program. This is elementary stuff#2022-05-2915:14kennytilton"Subscribe to each atom once"
Could HF/Missionary automagically detect multiple subscriptions?
Cells does not support cyclical dependency, ut they can be coded. If they are, the runtime cycle is detected by Cells. This does happen to me every six months or so, and indeed I am able to find some sloppy thinking and refactor the way it should have been in the first place. No more cycle.
The question I am raising to no avail is whether only elementary HF applications, and trivial test cases, can stay within the subscribe-once constraint. In my twenty-five years experience of deeply reactive systems, my prediction is "no", but quite a bit of HF code will be OK. Some of it will require a simple Staltz-ian refactoring.
The FB unread message count? Could force users to roll their own anti-multi-subscribe safety net. Hopefully not Redux. :)#2022-05-2915:14Dustin Getz> Some of it will require a simple Staltz-ian refactoring.
It will not, unless someone finds an actual bug in the propogation engine, in which case we will fix the bug#2022-05-2915:16Dustin Getz> Could HF/Missionary automagically detect multiple subscriptions?
We could yes, we do not choose to do this today. Our mission is to make it possible to express correct programs. It is not part of our mission to automatically turn buggy programs into correct programs. Fix your bug!#2022-05-2915:17Dustin Getz> In my twenty-five years experience of deeply reactive systems, my prediction is "no", but quite a bit of HF code will be OK. Some of it will require a simple Staltz-ian refactoring.
The burden of proof for continuing to make comments like this is to produce a test case. You did produce a pentagram of death test case (thank you), which we aced.#2022-05-3011:55Daniel JompheSince I'm the one who asked if this discussion had reached its fixpoint, I feel a need to come back to it a bit.
1. I very much want to thank both of you, Kenny and Dustin, for trying out as best as you could to answer my question in this space and stretch a bit more to make your point understood!
2. Then I want to pause and recognize how hard it can sometimes be on each viewpoint holder, when we feel we need to make sure our angle is understood and well considered. Especially when we know these things will be read again and again through the coming years.
3. Finally, reading you, I felt we share common sentiments on this: we all strive for the best. That's very clear to me. And therefore I wish that having discussed this now might lead to the overall best. And I'm already glad that Photon's and/or Missionnary's docs have been or will be augmented thanks to this exchange.#2022-05-3101:45kennytiltonHa-ha, I was wondering where you had gone to, @U0514DPR7! What did you learn from the exchange? Not the kumbaye my lord feel-good crap, I mean about how Missionary works. Think you can pass a review quiz?#2022-05-3116:03Daniel Jomphe@U0PUGPSFR, the answer reduces to "no, I'd have to invest in learning the ins and outs of the thing".
Here's a NLE;GTR (complement of TL;DR):
• After CL, I went to Haskell and Qi-then-Shen (both to stretch my mind) and Clojure (hoping I could one day apply it at work, which is now the case). And also to learn English better. And obviously the imperatives of my day jobs through the years.
• Although I keep a close watch on Hyperfiddle (HF), I haven't yet started doing the same on Missionary (M). I'm still at the stage of feeling M is unreadable, for not having invested in learning it. I know it's expected. I'm curious if I'll ever justify learning its usage. I'm more in pull-based learning mode than in push-based since I started being in the business of having 5 children.
• Although I've kept a watch on React and its various clojure wrappers, I haven't invested in the fundamentals of FRP and reactive programming. I thus don't feel qualified to judge M.
• What I keep from your exchange for now is that the current implementation will always converge to a coherent fixpoint but if our code calls code that instantiates more than one watch on the same atom and some 'parallel' intermediaries fire side effects, then we might sometimes observe incoherent side effects fired. Thus M (nor HF) doesn't protect us from that and we should pay attention to make sure no more than one watch per atom is instantiated.
• I don't have a practical intuition to determine in advance if it's going to be hard or easy to guard the number of watches per atom. Your experience seems to suggest this can easily get off of hand. And even though we can build quite complex systems even with this caveat, you strongly suggest it's important to add this check box to the fundamentals now. I don't know if it's practical or possible to add that to M, or possible to add that to HF as a kind of higher-level supervision. If I understand well, there's a purity constraint here, a bit like adding Cut to a Prolog makes it impure, and Dustin feels M should be kept pure (which feels like a noble argument, obviously, although one might judge that to be impractical in a grander scheme of things - I'm, again, not yet inclined to judge on this).
• I'm still wondering what parts of your vast and hard-won wisdom with Reactive Programming (and FRP, or not?) might bring benefits to either or both M and HF. For example if your glitch claims are really a problem of M's current implementation or not, or of HF on top of M only, or even of M or HF on top of clojure's current watch implementation (which I believe we shouldn't expect to change, unless...).
• I see the ravages of such glitches in some redux-based react apps, and also I've seen the chat bug in recent years on Fb. (BTW Facebook created Flux, not Redux, although they now employ Redux's creator.)
• I also wondered if by "kumbaye etc." you were targeting my response written yesterday, hoping to prevent me from (ever) writing another one like that, and/or triggering me to really engage in the argument for its worthy sake. ("ok Dan, you wrote enough human-political sweetening stuff, let's get back to the argument at hand because we can still learn and win more if we continue it".) If that's the case, I hope this answer is more favorable to you than my previous one.#2022-05-3116:30kennytilton"BTW Facebook created Flux, not Redux...." Correct, though Redux was created to fix Flux and did not change the fundamental design idea of exiling state to a secondary store. But good catch.
The rest of your summary shows excellent appreciation of the thread.#2022-06-0209:10leonoelTo complement dustin's answer, let me add some technical context about what could be the source of confusion here.
• Rx streams are discrete time only. Operators with continuous time semantics like combineLatest are necessarily glitchy, we all agree glitches are bad and stalz-refactoring is not an acceptable solution, no argument here. The problem is essential to the GoF observer pattern, clojure watch mechanism has the exact same problem.
• Matrix cells are a continuous time abstraction. The engine is able to maintain a DAG of cells with dynamic topology, and propagate a change through it without glitches. Discrete time events are not managed, so all effects interacting with the engine must be written imperative style, with callbacks and direct mutation of reference objects.
• Missionary aims to support both discrete and continuous time, to combine the benefits of managed effects AND glitch-free propagation. The flow abstraction supports both semantics and you can switch from one to the other with functional composition. This approach scales extremely well, you can go a very long way without having to ever mutate a single reference. Photon inherits this property.
@U0PUGPSFR The purpose of watch is purely interop. It is provided as a convenience for setting up a simple communication channel, to write unit tests or experiment at the REPL. Atoms and watches are sparingly used in a real application. There are some very specific use cases that require it, like representing cycles, and doing so requires special care anyway because then you need to ensure the propagation loop eventually converges. Here the atom is local state, it's watched immediately and you definitely don't want to expose it. In practice, the problem you're describing is really a non-issue if you understand what you're doing. We may reconsider this choice in the future if newcomers happen to struggle with it but as Dustin said this is a cosmetic change.#2022-06-0211:51kennytilton"you can go a very long way without having to ever mutate a single reference"
How does anything ever happen? How does a chess app player move P-K4? The question at hand is state management--if we are not mutating references, what is there to manage? Confused.
Anyway, all I had to work from were simple examples from my HF onboarding and Missionary tutorials. Those let me play a little and set up that glitchy example. Now I learn that I was not really using Missionary, I was doing sth else. OK.
So fine. I will guess that I misunderstood the idea that Missionary apps never change, and that they do process inputs. Is there a working example of realistic Missionary code that shows a new input propagating thru a DAG and obeying these constraints:
• all derived values are updated; (duh :))
• no derived values are updated more than once; (we need some way to count/observe updates);
• corollary: derivations never see obsolete values, defined as values that need to be recomputed because of the same change being processed.
Mind you, that ^^^ presumes a model David Gelernter christened a "trellis", one in which everything moves forward in discrete steps as determined by some arbitrary "generation" ID. Perhaps that does not apply to Missionary?#2022-06-0211:59Dustin GetzIf you're interested, pull latest – there are more and better examples now. I think you were the second person I onboarded (due to your level of expertise), so what you saw was super early. I demo with real web app examples now#2022-06-0212:00Dustin Getz~{:tag :a, :attrs {:href "https://github.com/hyperfiddle/photon/tree/master/src-docs/user"}, :content ("https://github.com/hyperfiddle/photon/tree/master/src-docs/user")}~
https://github.com/hyperfiddle/photon/blob/be908870b5cf43d8fd7f003488e632a4d091994c/src-docs/user/demo_webview.cljc#L46 single watch on database#2022-06-0212:00kennytiltonbtw, The RoboCup example involved an app that had to play soccer by
• receiving every 100ms a complete visual inventory of landmarks, other players, and the ball as a single JSON blob; and
• responding with actions such as turn, run, and kick.
So all dependency chains led back to the single visual inventory input fed from a handler on a UDP socket.#2022-06-0212:01kennytiltonThx, Dustin! I will check those out. 🙏#2022-06-0212:02Dustin GetzThis dom/input example doesn't even have a userland watch, the dom event subscription is abstracted by the input component https://github.com/hyperfiddle/photon/blob/be908870b5cf43d8fd7f003488e632a4d091994c/src-docs/user/demo_system_properties.cljc#L17 (there is a watch-like construct inside, m/observe which is how you implement m/watch)#2022-06-0212:07Dustin GetzI would love to try robocup when we get time, maybe an opportunity to collaborate#2022-06-0215:32kennytiltonbtw, RoboCells was in Common Lisp. And the league itself seems retired as of 2017. Lemme look on SourceForge...#2022-06-0215:46kennytiltonThat was easy. https://sourceforge.net/projects/robocells/
Caveat: I remember doing one such archaeological dig and coming up with an old version, so this may not be the version that finally triggered a breaking glitch.#2022-06-0215:53kennytiltonRoboCup for real bots seems quite active, but I do not see a simulation league still active: https://www.robocup.org/#2022-05-1312:42Dustin Getz#2022-05-1313:17gravHi @dustingetz - any onboarding invites left? 🙂#2022-05-1313:19Dustin GetzAdded you to the next wave, I'll reach out soon!#2022-05-1313:24maxtI would like one to!#2022-05-1313:24maxtplease 🙂#2022-05-1313:26Dustin GetzAdded you to the list, what's your full name? A website or linkedin link would help as well so I have more context#2022-05-1313:29maxtI'll PM you#2022-05-1313:35Mert NuhogluHi @dustingetz
I am a newcomer to Clojure. I am still studying some Clojure tools. Currently, I have been focusing into Datomic/Datalog libraries. Next, I will start studying Missionary.
If having a newcomer might be useful for testing Photon, I would be more than happy to participate in the on-boarding program. I have a lot of free time currently and in the next months. So, I am willing to dedicate a good of amount of it to studying and developing Photon apps.
https://mertnuhoglu.com/
https://www.linkedin.com/in/mertnuhoglu/
https://github.com/mertnuhoglu/#2022-05-1313:49Dustin GetzI'll add you to the list!#2022-05-1313:51Mert NuhogluThank you very much. Appreciated :thumbsup:#2022-05-1314:39Debdut KarmakarHey @dustingetz can I get an invite ps?🥹#2022-05-1515:07Dustin GetzDMed#2022-05-1321:27braai engineerHow do I get an invite, plz?#2022-05-1515:07Dustin GetzDMed#2022-05-1423:06Benjamin CI am also interested in an invite. 🙂
Partly because I would like to try it for building my own website, and partly because I am really curious to see rcf/test in action!#2022-05-1515:07Dustin GetzDMed#2022-05-1620:51Jose Vareladitto#2022-05-1716:47Dustin Getzfullstack web view with query/view composition and render, database on server. With stable hot code reloading#2022-05-1718:39Peter Nagylines 55 and 56 both use {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "5f211f"}, :content ("[email protected]")} but we're not in a backticked form. I guess you're reusing the operators in p/defn compiler for something?#2022-05-1718:40Dustin Getz/cdn-cgi/l/email-protection outside of ` is the current syntax for client/server transfer, we have not gotten to implementing the better (p/client) syntax yet#2022-05-1718:58Peter Nagyyou mean the p/client and p/server code samples you showed where wishful thinking?#2022-05-1719:00Dustin GetzTo communicate clearly the talk uses our target syntax as the /cdn-cgi/l/email-protection was a mistake and is not readable, we should have the good syntax soon#2022-05-1719:00Peter Nagyshould I read it as a toggle? Like toplevel starts as cljs, first {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "196759"}, :content ("[email protected]")} toggles to server, next {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "463806"}, :content ("[email protected]")} toggles to client again?#2022-05-1719:00Dustin Getzyes it's a toggle currently#2022-05-1719:01Peter Nagyunderstood, thanks#2022-05-2713:10Daniel JompheMy colleagues want a quick demo of photon. I improved the webview example with state-preserving hot-reloading of the server db.
I had to wrap the reader conditionals in e.g. do or let exprs, otherwise e.g. defonce thought I passed it only 1 arg, the name conn.
(defonce conn (let [c (do #?(:clj (d/create-conn {:order/email {}})))]
#?(:clj (d/transact! c [{:order/email "#2022-05-2713:13Daniel JompheI tried defonce-ing the !state atom of the client too, hoping that the input would preserve its email filter value. No luck there.
Anyway this'll already be a very nice demo!
@U2DART3HA I think I should ping you about this small improvement to the demo(s).
If you didn't get what I mean by all this, I'll screencast a 1 minute demo for you, showing before & after the change.#2022-05-2713:33Dustin GetzUnderstood, will push the fix in a minute#2022-05-2713:34Daniel JompheWill pull in days, unless other interesting stuff surfaces. 🙂 No urgency but thanks a lot!#2022-05-1810:46Dustin GetzOld DAG visualizer from early 2021#2022-05-1810:56delaguardohi! Do you still have invites available? I'm interested to play with hyperfiddle#2022-05-1811:59Dustin GetzThanks for asking, Added you to the list & DMing you#2022-05-1914:48Peter Nagywould it be possible to define p/server and p/client as a macro that resolves to just {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d3ad93"}, :content ("[email protected]")}? It would be a bit of a lie but could allow to write future-proof code for the moment you implement these for real#2022-05-1914:48Dustin GetzThe semantics are different, /cdn-cgi/l/email-protection is a toggle. I do not recommend this#2022-05-1914:49Dustin GetzIt will be a very easy refactor in the long run, as it is very obvious from looking at a piece of code, where it is meant to run#2022-05-1914:57Peter Nagyalright, thanks#2022-05-1919:12Dustin GetzAlpha testers please pull latest, lots of fixes, much improved entrypoint, fast way to switch between example apps at the repl, see readme. Warning there is a build step now#2022-05-1919:23Dustin Getz#2022-05-1922:19Dustin GetzTwo Clocks#2022-05-2009:52henrikCool! You made the example!#2022-05-2012:50Geoffrey Gaillard@a.grison#2022-05-2015:29lilactownI think I have some ideas how to make the project easier to get running using emacs#2022-05-2015:29lilactownwhere's a good place for me to put a bit of code?#2022-05-2016:19Dustin GetzYou are on latest master right? The entrypoint changed a lot to hopefully make this easier#2022-05-2016:19Dustin Getzwe're pushing every day#2022-05-2016:20Dustin GetzYou can make a public github repo, it's just you'll need to manually {:vendor/local "../photon"} the private dep#2022-05-2016:21Dustin GetzThere is also https://github.com/hyperfiddle/photon-starter-app which uses the same relative path dep#2022-05-2016:29lilactownI have some photon-project-specific emacs config. I'll open a PR#2022-05-2019:10lilactownI ran into an issue when trying to do mutual recursion between p/defns. example here https://github.com/lilactown/photon-tree#2022-05-2019:15Dustin Getzoh yes, it's not implemented yet, it can be hacked by using dynamic scope to bind p/fn closures#2022-05-2019:16Dustin GetzI will try to provide an example demonstrating#2022-05-2019:29Dustin Getz[better example moved to channel as attachment]#2022-05-2019:30Dustin Getzpasses#2022-05-2019:34Dustin Getz#2022-05-2213:59Dustin GetzThings that can be built in Photon today:
• Clojure docstring search UI and live namespace explorer
• clojure spec registry browser with playground
• OS process viewer and killer
• /etc/hosts editor
• folder explorer (including google drive search UI that is faster than google drive)
• git history viewer
• datomic/datascript etc history viewer and transaction browser
• reactive REPL
All this as easy as writing PHP
If you would like to spend a weekend trying to build one of these, reply here and skip the waitlist to be onboarded this week#2022-05-2309:14nottmeythe datomic history/structure/transaction browser sounds nice, would’ve tried that anyway though#2022-05-2620:17Benjamin CDoes Photon currently require you to use ClojureScript as the client? For example, could you use cljfx or HumbleUI for a desktop client to make a OS process viewer and killer of a running server?#2022-05-2701:55Dustin Getzno technical barriers, photon lang tests pass in single process configurations (i.e. jvm only or cljs only). as far as UI platform drivers we only have DOM today but likely can be ported to any platform, photon-dom is 200 LOC#2022-05-2702:19Benjamin CAwesome. This is really exciting!#2022-05-2718:10nottmeyI’ll be working on the datomic viewer here: https://github.com/nottmey/photon-datomic-viewer#2022-05-2215:37ribelo@dustingetz#2022-05-2300:37PanelI like the sound of reactive REPL... clojure-spin#2022-05-2608:22Tomasz KalisiakHi, I know I come late but I would love to join the waitlist for Photon invites. : )#2022-05-2701:17Dustin Getzi will DM!#2022-05-2715:07Daniel JomphePhoton is getting too much love these days ☺️, allow me to bring RCF in focus just a bit.
Here's me wondering if it would be fine for RCF tests forms to print a newline after their last test?#2022-05-2717:11Geoffrey GaillardThis is welcome 😄
The concept of a "last test" in RCF is not clear.
An RCF test block is just an expression.
Adding a newline at the end of each test could be is a solution (in the reporter)
Using a test runner is another one.#2022-05-2720:47Dustin Getzyeah agree this needs fixing but not sure how. we do have monkey patches that hook into the module loader in clj and cljs that we could reconsider#2022-05-2718:50nottmeyWhen I do this: (adding com.datomic/dev-local)
{:paths ["src" "resources"]
:deps {ch.qos.logback/logback-classic {:mvn/version "1.2.11"}
ch.qos.logback/logback-core {:mvn/version "1.2.11"}
com.datomic/client-api {:mvn/version "1.0.58"}
hyperfiddle/photon {:local/root "../photon"}
org.clojure/clojure {:mvn/version "1.11.1"}
org.clojure/clojurescript {:mvn/version "1.11.4"}
thheller/shadow-cljs {:mvn/version "2.18.0"}}
:aliases {:dev {:extra-deps {com.datomic/dev-local {:mvn/version "1.0.243"}}
:exec-fn user/main
:exec-args {}}}}
I get a com.google.javascript.jscomp.deps.ModuleLoader Error … (see thread)
This seems like a ClojureScript Error, but com.datomic/dev-local should be a Server-Side only dependency.
How do I configure it to only be included in the server side?#2022-05-2718:50nottmeyException in thread "main" Syntax error macroexpanding at (closure.clj:77:5).
at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1750)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3713)
at clojure.lang.Compiler$DefExpr.eval(Compiler.java:457)
at clojure.lang.Compiler.eval(Compiler.java:7199)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:551)
at shadow.build.js_support$eval14638$loading__6789__auto____14639.invoke(js_support.clj:1)
at shadow.build.js_support$eval14638.invokeStatic(js_support.clj:1)
at shadow.build.js_support$eval14638.invoke(js_support.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:619)
at shadow.build.resolve$eval12786$loading__6789__auto____12787.invoke(resolve.clj:1)
at shadow.build.resolve$eval12786.invokeStatic(resolve.clj:1)
at shadow.build.resolve$eval12786.invoke(resolve.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:2793)
at shadow.build.api$eval12778$loading__6789__auto____12779.invoke(api.clj:1)
at shadow.build.api$eval12778.invokeStatic(api.clj:1)
at shadow.build.api$eval12778.invoke(api.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:2793)
at shadow.build$eval12407$loading__6789__auto____12408.invoke(build.clj:1)
at shadow.build$eval12407.invokeStatic(build.clj:1)
at shadow.build$eval12407.invoke(build.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:3204)
at shadow.cljs.devtools.api$eval12270$loading__6789__auto____12271.invoke(api.clj:1)
at shadow.cljs.devtools.api$eval12270.invokeStatic(api.clj:1)
at shadow.cljs.devtools.api$eval12270.invoke(api.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.load(RT.java:459)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6908.invoke(core.clj:6161)
at clojure.core$load.invokeStatic(core.clj:6160)
at clojure.core$load.doInvoke(core.clj:6144)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5933)
at clojure.core$load_one.invoke(core.clj:5928)
at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
at clojure.core$load_lib.invokeStatic(core.clj:5974)
at clojure.core$load_lib.doInvoke(core.clj:5953)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$load_libs.invokeStatic(core.clj:6016)
at clojure.core$load_libs.doInvoke(core.clj:6000)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$require.invokeStatic(core.clj:6038)
at clojure.core$require.doInvoke(core.clj:6038)
at clojure.lang.RestFn.invoke(RestFn.java:482)
at user$eval138$loading__6789__auto____139.invoke(user.clj:1)
at user$eval138.invokeStatic(user.clj:1)
at user$eval138.invoke(user.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7183)
at clojure.lang.Compiler.load(Compiler.java:7653)
at clojure.lang.RT.loadResourceScript(RT.java:381)
at clojure.lang.RT.loadResourceScript(RT.java:368)
at clojure.lang.RT.maybeLoadResourceScript(RT.java:364)
at clojure.lang.RT.doInit(RT.java:486)
at clojure.lang.RT.init(RT.java:467)
at clojure.main.main(main.java:38)
Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Collector com.google.common.collect.ImmutableSortedSet.toImmutableSortedSet(java.util.Comparator)'
at com.google.javascript.jscomp.deps.ModuleLoader.createRootPaths(ModuleLoader.java:257)
at com.google.javascript.jscomp.deps.ModuleLoader.<init>(ModuleLoader.java:147)
at com.google.javascript.jscomp.deps.ModuleLoader.<init>(ModuleLoader.java:48)
at com.google.javascript.jscomp.deps.ModuleLoader$Builder.build(ModuleLoader.java:139)
at com.google.javascript.jscomp.deps.ModuleLoader.<clinit>(ModuleLoader.java:408)
at com.google.javascript.jscomp.DiagnosticGroups.<clinit>(DiagnosticGroups.java:182)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1743)
... 178 more
#2022-05-2718:55Geoffrey GaillardMy first thought is a closure compiler dependency version conflict. This hypothesis can be verified by looking at the output of clj -A:dev -Stree#2022-05-2718:56nottmey➜ clj -A:dev -Stree
org.clojure/clojure 1.11.1
. org.clojure/spec.alpha 0.3.218
. org.clojure/core.specs.alpha 0.2.62
ch.qos.logback/logback-classic 1.2.11
X ch.qos.logback/logback-core 1.2.11 :use-top
. org.slf4j/slf4j-api 1.7.32
ch.qos.logback/logback-core 1.2.11
hyperfiddle/photon /Users/m/Projects/photon
. org.eclipse.jetty/jetty-servlet 9.4.44.v20210927
. org.eclipse.jetty/jetty-security 9.4.44.v20210927
. org.eclipse.jetty/jetty-server 9.4.44.v20210927
. org.eclipse.jetty/jetty-util-ajax 9.4.44.v20210927
. org.eclipse.jetty/jetty-util 9.4.44.v20210927
. org.clojure/tools.logging 1.2.4
. org.clojure/tools.analyzer.jvm 1.2.2
. org.clojure/tools.analyzer 1.1.0
. org.clojure/core.memoize 1.0.253
. org.ow2.asm/asm 9.2
. org.clojure/tools.reader 1.3.6
. org.hsluv/hsluv 0.2
. com.cognitect/transit-cljs 0.8.269
. com.cognitect/transit-js 0.8.874
X org.clojure/clojurescript 1.11.4 :use-top
. org.eclipse.jetty.websocket/websocket-api 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-servlet 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-api 9.4.44.v20210927
. javax.servlet/javax.servlet-api 3.1.0
. com.cognitect/transit-clj 1.0.329
. com.cognitect/transit-java 1.0.362
. com.fasterxml.jackson.core/jackson-core 2.8.7
. org.msgpack/msgpack 0.6.12
. com.googlecode.json-simple/json-simple 1.1.1
. org.javassist/javassist 3.18.1-GA
. javax.xml.bind/jaxb-api 2.3.0
. missionary/missionary b.27-SNAPSHOT
X org.clojure/clojurescript 1.10.879 :use-top
. org.reactivestreams/reactive-streams 1.0.3
. cloroutine/cloroutine 10
X org.clojure/clojurescript 1.10.520 :use-top
X org.clojure/tools.analyzer.jvm 0.7.2 :older-version
. nextjournal/clojure-mode 699a445
. applied-science/js-interop 0.2.5
X org.clojure/clojurescript 1.10.866 :use-top
. hyperfiddle/rcf bcd9900
. datascript/datascript 1.3.12
. persistent-sorted-set/persistent-sorted-set 0.1.4
. io.replikativ/datahike 0.4.1491
. io.lambdaforge/datalog-parser 0.1.11
. io.replikativ/hitchhiker-tree 0.1.11
X org.clojure/clojurescript 1.10.773 :use-top
. com.taoensso/carmine 2.20.0
X com.taoensso/encore 2.127.0 :older-version
X com.taoensso/timbre 4.10.0 :older-version
. com.taoensso/nippy 2.14.0
X org.clojure/tools.reader 1.1.1 :older-version
X com.taoensso/encore 2.93.0 :older-version
. org.iq80.snappy/snappy 0.4
. org.tukaani/xz 1.6
. net.jpountz.lz4/lz4 1.3
. org.apache.commons/commons-pool2 2.4.2
X commons-codec/commons-codec 1.13 :older-version
X org.clojure/core.rrb-vector 0.1.1 :older-version
X org.clojure/core.memoize 1.0.236 :older-version
X org.clojure/core.cache 1.0.207 :older-version
. io.replikativ/konserve 0.5.1
. io.replikativ/incognito 0.2.5
. fress/fress 0.3.1
X org.clojure/clojurescript 1.10.339 :use-top
. org.clojure/data.fressian 0.2.1
X org.clojure/core.async 0.4.490 :older-version
. org.clojure/data.fressian 0.2.1
. org.fressian/fressian 0.6.6
. io.replikativ/hasch 0.3.5
X io.replikativ/incognito 0.2.2 :older-version
. org.clojure/data.codec 0.1.1
. org.clojars.mmb90/cljs-cache 0.1.4
X org.clojure/core.async 0.2.385 :older-version
X org.clojure/clojurescript 1.9.89 :use-top
. adzerk/boot-test 1.1.2
. crisptrutski/boot-cljs-test 0.2.2-20160402.204547-3
. tailrecursion/cljs-priority-map 1.2.1
X org.clojure/clojurescript 1.7.170 :use-top
. mvxcvi/clj-cbor 1.1.0
X org.clojure/clojurescript 1.11.4 :use-top
. io.replikativ/zufall 0.1.0
. environ/environ 1.2.0
. com.taoensso/timbre 5.2.1
. com.taoensso/encore 3.21.0
. org.clojure/tools.reader 1.3.6
. com.taoensso/truss 1.6.0
. io.aviso/pretty 1.1.1
. persistent-sorted-set/persistent-sorted-set 0.1.4
. junit/junit 4.13.2
. org.hamcrest/hamcrest-core 1.3
. org.clojure/tools.reader 1.3.6
. io.replikativ/superv.async 0.3.43
X org.clojure/clojurescript 1.11.4 :use-top
. org.clojure/core.async 1.5.648
. org.eclipse.jetty.websocket/websocket-server 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-common 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-api 9.4.44.v20210927
. org.eclipse.jetty/jetty-util 9.4.44.v20210927
. org.eclipse.jetty/jetty-io 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-client 9.4.44.v20210927
. org.eclipse.jetty/jetty-client 9.4.44.v20210927
. org.eclipse.jetty/jetty-http 9.4.44.v20210927
. org.eclipse.jetty/jetty-io 9.4.44.v20210927
. org.eclipse.jetty/jetty-util 9.4.44.v20210927
. org.eclipse.jetty/jetty-io 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-common 9.4.44.v20210927
. org.eclipse.jetty.websocket/websocket-servlet 9.4.44.v20210927
. org.eclipse.jetty/jetty-servlet 9.4.44.v20210927
. org.eclipse.jetty/jetty-http 9.4.44.v20210927
. org.eclipse.jetty/jetty-util 9.4.44.v20210927
. org.eclipse.jetty/jetty-io 9.4.44.v20210927
. org.eclipse.jetty/jetty-server 9.4.44.v20210927
. javax.servlet/javax.servlet-api 3.1.0
. org.eclipse.jetty/jetty-http 9.4.44.v20210927
. org.eclipse.jetty/jetty-io 9.4.44.v20210927
. org.eclipse.jetty/jetty-util 9.4.44.v20210927
org.clojure/clojurescript 1.11.4
X com.google.javascript/closure-compiler-unshaded v20210808 :superseded
. org.clojure/google-closure-library 0.0-20211011-0726fdeb
. org.clojure/google-closure-library-third-party 0.0-20211011-0726fdeb
X org.clojure/data.json 0.2.6 :superseded
X org.clojure/tools.reader 1.3.3 :superseded
X com.cognitect/transit-clj 0.8.309 :older-version
thheller/shadow-cljs 2.18.0
. org.clojure/data.json 2.4.0 :newer-version
. org.clojure/tools.cli 1.0.206
. org.clojure/tools.reader 1.3.6 :newer-version
. nrepl/nrepl 0.9.0
. cider/piggieback 0.5.3
. com.cognitect/transit-clj 1.0.329
. com.cognitect/transit-cljs 0.8.269
. org.clojure/core.async 1.5.648
. org.clojure/tools.analyzer.jvm 1.2.2
X org.clojure/clojurescript 1.11.4 :use-top
. com.google.javascript/closure-compiler-unshaded v20220301 :newer-version
. org.clojure/google-closure-library 0.0-20211011-0726fdeb
. org.clojure/google-closure-library-third-party 0.0-20211011-0726fdeb
. thheller/shadow-util 0.7.0
. hiccup/hiccup 1.0.5
. thheller/shadow-client 1.3.3
X org.clojure/core.async 0.3.443 :older-version
. hiccup/hiccup 1.0.5
. thheller/shadow-undertow 0.2.0
X org.clojure/core.async 1.3.610 :older-version
. io.undertow/undertow-core 2.2.4.Final
. org.jboss.logging/jboss-logging 3.4.1.Final
. org.jboss.xnio/xnio-api 3.8.0.Final
. org.wildfly.common/wildfly-common 1.5.2.Final
. org.wildfly.client/wildfly-client-config 1.0.1.Final
X org.jboss.logging/jboss-logging 3.3.1.Final :older-version
X org.wildfly.common/wildfly-common 1.2.0.Final :older-version
. org.jboss.xnio/xnio-nio 3.8.0.Final
. org.jboss.xnio/xnio-api 3.8.0.Final
X org.jboss.threads/jboss-threads 2.3.3.Final :older-version
. org.jboss.threads/jboss-threads 3.1.0.Final
. org.jboss.logging/jboss-logging 3.4.1.Final
. thheller/shadow-cljsjs 0.0.22
. hiccup/hiccup 1.0.5
. ring/ring-core 1.9.5
. ring/ring-codec 1.1.3
. commons-codec/commons-codec 1.15
. commons-io/commons-io 2.10.0
. commons-fileupload/commons-fileupload 1.4
X commons-io/commons-io 2.2 :older-version
. crypto-random/crypto-random 1.2.1
. commons-codec/commons-codec 1.15
. crypto-equality/crypto-equality 1.0.0
. io.methvin/directory-watcher 0.15.1
. net.java.dev.jna/jna 5.7.0
X org.slf4j/slf4j-api 1.7.30 :older-version
. expound/expound 0.9.0
. fipp/fipp 0.6.25
. org.clojure/core.rrb-vector 0.1.2
. com.bhauman/cljs-test-display 0.1.1
com.datomic/dev-local 1.0.243
. com.datomic/client 1.0.126
. com.cognitect/anomalies 0.1.12
. com.datomic/client-api 1.0.58
. com.datomic/client-impl-shared 1.0.98
. com.cognitect/hmac-authn 0.1.210
. commons-codec/commons-codec 1.15
. com.cognitect/anomalies 0.1.12
. org.clojure/core.async 1.5.648
X com.cognitect/http-client 1.0.111 :excluded
. com.cognitect/transit-clj 1.0.329
. com.datomic/query-support 0.8.28
. org.clojure/core.async 1.5.648
. com.cognitect/anomalies 0.1.12
. com.datomic/client-api 1.0.58
. org.clojure/core.async 1.5.648
. com.datomic/client-impl-shared 1.0.98
. com.cognitect/anomalies 0.1.12
. org.clojure/core.async 1.5.648
. com.datomic/query-support 0.8.28
. org.clojure/core.async 1.5.648
. org.clojure/tools.analyzer.jvm 1.2.2
. org.clojure/tools.analyzer 1.1.0
. org.clojure/core.memoize 1.0.253
. org.clojure/core.cache 1.0.225
. org.clojure/core.cache 1.0.225
. org.clojure/data.priority-map 1.1.0
. org.clojure/data.priority-map 1.1.0
. org.ow2.asm/asm 9.2
. org.clojure/tools.reader 1.3.6
X org.clojure/spec.alpha 0.1.143 :older-version
X org.clojure/core.specs.alpha 0.1.24 :older-version
. com.google.guava/guava 31.0.1-android
. com.google.guava/failureaccess 1.0.1
. com.google.guava/listenablefuture 9999.0-empty-to-avoid-conflict-with-guava
. com.google.code.findbugs/jsr305 3.0.2
. org.checkerframework/checker-qual 3.12.0
. org.checkerframework/checker-compat-qual 2.5.5
. com.google.errorprone/error_prone_annotations 2.7.1
. com.google.j2objc/j2objc-annotations 1.3
. com.google.guava/failureaccess 1.0.1
. com.google.guava/listenablefuture 9999.0-empty-to-avoid-conflict-with-guava
. com.google.code.findbugs/jsr305 3.0.2
. org.checkerframework/checker-qual 3.12.0
. org.checkerframework/checker-compat-qual 2.5.5
. com.google.errorprone/error_prone_annotations 2.7.1
. com.google.j2objc/j2objc-annotations 1.3
. org.fressian/fressian 0.6.6#2022-05-2718:59nottmeyIf it’s a version conflict, I don’t see it 😅#2022-05-2720:42Dustin Getzcan you show us the app file?#2022-05-2720:46nottmeyno changes: https://github.com/nottmey/photon-datomic-viewer/blob/main/src/app/core.cljc#2022-05-2720:47nottmeybasically it’s https://github.com/nottmey/photon-datomic-viewer only com.datomic/dev-local is uncommented in the deps file#2022-05-2723:05nottmey(sadly this is blocking me in implementing the database access 😕)#2022-05-2723:14Dustin Getzi'm afk and the team is asleep- i will take a look first thing tomorrow morning#2022-05-2723:41Dustin GetzI've reproduced it#2022-05-2723:49Dustin GetzI removed all Photon dependencies and can still reproduce it, this is a Datomic issue#2022-05-2800:05Dustin Getzit's a shadow issue; fix is to force Guava to latest
com.google.guava/guava {:mvn/version "31.1-jre"} ; force guava to latest to fix shadow issue
#2022-05-2807:21nottmeyOh wow 🙈#2022-05-2807:28nottmeyfrom the tree, I would never have guessed
sorry that I didn’t check in that direction
thank you very much for the find! (works)#2022-05-2723:03nottmeyPhoton: I tried to create a simple routing concept (client side):
(p/defn App []
(let [!route (atom :home)
route (p/watch !route)]
(dom/div
(condp = route
:home (HomeScreen. !route)
:other (OtherScreen.)))))
In HomeScreen I simply wanted to (reset! !route :other) . But I noticed, how do I listen on button clicks (`(dom/events dom/parent "click")`) and make it call a function? Or is there a better way of doing this?#2022-05-2723:32Dustin GetzThis pattern works for now (there is probably a better way to do it though)#2022-05-2807:29nottmeyAlright, I’ll try that#2022-05-2809:22nottmeyworks like a charm#2022-05-2811:09nottmeyPhoton Observation: The sorting of maps ist lost here. This seems to be a behavior of dom/for .
(p/defn DataViewer [title maps]
(let [ks (u/all-keys maps)]
(dom/h1 (dom/text title))
(dom/table
(dom/thead
(dom/for [k ks]
(dom/td (dom/style {"min-width" "5em"}) (dom/text k))))
(dom/tbody
(dom/for [m maps]
(dom/tr
(dom/for [k ks]
(dom/td (dom/text (m k))))))))))#2022-05-2811:17Dustin Getzthanks - this is fixed in a feature branch and in test, landing asap#2022-05-2912:54Dustin GetzDatomic transaction browser, contributed by @nottmey. he said it took 10 hours to make (and it was his first attempt to make a Photon app). Thank you Malte!#2022-05-2918:59nottmeyin hindsight I think this is doable in 2 hours of fiddeling, the delay was just learning and dealing with bugs you will never have again 😄#2022-05-3020:54genekimThis is so cool!!! Congrats @nottmey , and I'm so excited to see this coming together, @U09K620SG
(In a couple of weeks, I'd love to sign up for one of the projects. It's an extremely busy time for me right now, but this looks so compelling!)#2022-05-3112:34Dustin Getz@U899JBRPF should we try this with xtdb?#2022-05-3112:55nottmeywe could leave this as a small nice demo, or I could build a real tool out of it :thinking_face:
I guess this depends on how much positive feedback we get from cognitec, #datomic, and other datalog dbs#2022-05-3112:56nottmey@U09K620SG do you mind if I ask #datomic if they like it? ^^#2022-05-3113:17refset> @U899JBRPF should we try this with xtdb?
I would be in your debt! 🙏#2022-06-0111:11Dustin Getz@nottmey please wait to share until we fix the issues, because people are going to want to run it#2022-05-3008:49nottmeyPhoton Datomic Viewer: New Features & Better Structure ✨#2022-05-3112:32Dustin Getz@U3ZG4CAF8, @U2DART3HA is going to send you a PR with some very minor changes. It may take him a few days. (your app is great, i don't recall seeing any problems other than what you asked about already)
Geoffrey the known issues are:
• how to do local pagination
• can queries be pushed lower in the tree
• move to Async Datomic Client API (core.async interop)
• fix dom ordering in for loops
• for loop should start on server and send collection diffs on the wire
#2022-05-3112:53nottmeyAlright :thumbsup:#2022-05-3011:15Daniel JompheWe'll need to have someone design a quality logo & emoji to react to this great stuff in e.g. Slack. 😄 Looks like we're soon going to view an endless wave of tooling announcements.#2022-06-0109:32Adrian SmithHey can I request an invite? I'm abroad on holiday at the moment but definitely want to look at this when I'm back#2022-06-0113:27Dustin GetzYes! Added to the list. We're overloaded right now, I'll get back to you soon#2022-06-0109:33Adrian SmithThinking it'd be nice if there was a shared wiki for Clojure, trying hard to not reinvent the wiki wheel but would like it in Clojure#2022-06-0521:43bbloom👋#2022-06-1407:53Peter Nagyanyone familiar with this library? http://janusjs.org/ . Has some interesting ideas#2022-06-1415:23Peter Nagyif you are the watching kind https://www.youtube.com/watch?v=7S57O3VwIyQ#2022-06-1408:47nottmeyI recently read https://dev.to/tigt/making-the-worlds-fastest-website-and-other-mistakes-56na#fn1 (probably a lot of you did already)
Is photon also going to do HTML streaming? I think it fits very neatly with its computing model.
PS: Combined with edge computing, this would be amazingly fast. (possibly be the fastest you can go with clojurescript?)
-> we could skip the whole router thing and “simply” build a MPA (multi page app) again 😄#2022-06-1409:04Geoffrey GaillardPhoton shines with client-server distributed programs. The DAG being split optimally means it suffers less from low TTI than regular non-trivial SPAs.
That said, I share your enthusiasm a lot! We could have the best of both worlds.
HTML streaming makes sense in photon. We are not prioritizing it, but it should be easy enough to implement from userland.
Take photon-dom for instance: it creates, mounts and attaches nodes to the dom as a continuous flow.
Reverse it: instead of creating, mounting and attaching, return a formula for creating, mounting and attaching dom nodes. It’s fine grained and it’s already a tree of flows. Gather them all and serialize to HTML, then pipe it over the wire.
On the client you’ll have to insert HTML bits at the right place, so you’ll need a parent-id on each fragment. Which you get for free given the existing tree structure.#2022-06-1409:09nottmeyexactly, I see the differences, but even when targeting to build SPAs, these techniques sound very neat#2022-06-1411:28nottmeythat would also be something I would love to tinker with (so that photon can compile in that way)
but photon core is probably not stable enough yet, right?#2022-06-1412:38Geoffrey GaillardIf you want to start with streamed HTML then hydrate it to an SPA, then this is too early.
If you want to do streamed, server-side rendered HTML, with server-only state and user events handled on the server, then it can be done today. You would get something close to phoenix liveview:
state on the server and a thin client.
Ping me if you want to Zoom about it 🙂#2022-06-1412:40Dustin GetzMalte can you define precisely "HTML Streaming" as it can mean several different things, I can't easily find a clear definition in that blog series#2022-06-1415:04nottmeyI was planning on understanding that myself: In the series he talks about https://markojs.com/ and they refer to this blogpost https://dev.to/ryansolid/server-rendering-in-javascript-optimizing-performance-1jnk#2022-06-1415:06nottmeyin their docs they refer https://markojs.com/docs/why-is-marko-fast/#2022-06-1415:09nottmeyThis gives a rough understanding:
> When rendering on the server, Marko renders directly to a string representation of the document (HTML) that can be sent as the HTTP response.
>
> When rendering in the browser, an HTML string would have to be parsed in order to update the DOM. For this reason, Marko compiles a view to a program that renders directly to a virtual document (VDOM) tree that can be used to efficiently update the real DOM when targeting the browser.#2022-06-1415:25lilactownI've spent a bunch of time now investigating React's new streaming SSR. the demo's are pretty neat, but it's not something I've started using in anger yet.
the insight I've pulled from their demos and messing with it myself: it is all about latency. you have latency on the client fetching the JS, you have latency on the server fetching the data to know what HTML to render. all of these can affect both the time to first paint and the time to interactive.
the idea is that if you can organize the HTML you send to the client and then the HTML that becomes interactive to work in concert, you have a way to significantly increase the efficiency of the whole process and improve TTFP and TTI.
on the server, React starts by rendering the shell of the page, which kicks off any data fetching needed in each component. By using suspense boundaries, the server knows on that first render to have loading messages in place wherever data is being fetched. It then sends that shell (with script tags to load client code), and keeps the connection open.
as data resolves on the server, it then will send incremental updates to the parts of the page that are ready to be shown to the user. it sends both the new HTML + a tiny bit of JS to replace the fallback in the suspense boundary with the HTML. once the fallback has been replaced, the React client code can hydrate just that part of the page to make it interactive. it knows how to do this because each suspense boundary is uniquely identified by its place in the tree of react elements and in the DOM, and that is known to server & client.
I like this model a lot in theory, and it seems like it would be amenable to some of the architectural decisions photon has made. the part that I think is missing is the ability to track "loading" states, and the fact that the client/server contract (i.e. what HTML is rendered on the server can be properly hydrated on the client) isn't known yet#2022-06-1415:37nottmeyAs a sidenote to Dustin: Under “HTML streaming” I meant this general technique of allowing to have a fast TTFP and TTI, while nonblockingly sending html to the client as it becomes available.
I guess there are specific/additional techniques (maybe termed similarly) to optimize the rendering and transfer of the dom parts.#2022-06-1415:41Dustin GetzThanks – yeah it seems Photon can support this model, just need a slightly smarter DOM effect layer (today photon-dom is like 100 LOC). Does not even seem hard. In Photon the client/server are synchronized so partial rendering and loading states should work out for free IIUC.#2022-06-1513:17Dustin GetzHyperfiddle first peek:
UI Modeling language for CRUD apps ("HTML for UI")
+ hyperlinks
+ compiler managed datasync
+ WYSIWYG in-browser live editor#2022-06-1515:27Dustin GetzLonger end to end clip:#2022-06-1609:34Daniel JompheWhat does QL stand for in HFQL?#2022-06-1609:35Daniel JompheI keep reading Hyperfiddle Query Language.#2022-06-1610:43Dustin Getzthat's what it stands for, though we should revisit that#2022-06-2114:09Dustin GetzPhoton recursive tree view with compiler managed incremental datasync
(Not even clear what the "REST API" for something like this would even looks like, this is a higher level of expressiveness than is possible with current gen datasync methods)#2022-06-2114:11Dustin Getz{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "681628"}, :content ("[email protected]")} is today's syntax for client/server transfer points, it's a relative toggle between client and server. the better (p/client) (p/server) block syntax is on the roadmap#2022-06-2114:22maxtThat is very cool!#2022-06-2121:27chrisblomCool stuff!#2022-06-2312:46Dustin Getz#2022-12-1220:01Ben SlessHow much of Clojure does Photon support?#2022-12-1220:01Dustin Getz99.9% maybe#2022-12-1220:01Dustin Getzhand waving over todos#2022-12-1220:01Dustin Getzthere are some super interesting edge cases#2022-12-1220:02Ben SlessAsking because I'm theorizing the overhead of implementing all the basic operators as differentiable signals#2022-12-1220:02xificurCcase op
(ns ns* deftype* defrecord* var)
(throw (ex-info "Unsupported operation." {:op op :args args}))#2022-12-1220:03Dustin GetzI am specifically looking to understand a pure continuous time model, i think a system can be modeled in CT all the way up until you introduce transactions at which point you have db = App(db) (a discrete iteration) or alternately db = Σ [t=0..T] App(t)Δt (app is a reactive function that returns transaction directives over time, a log)#2022-12-1220:03Ben SlessContinuous time is hard, I'd just do discrete time#2022-12-1220:04Ben SlessIn continuous time you can have no partial ordering#2022-12-1220:04Ben SlessAnd good luck modelling / solving feedback loops#2022-12-1220:04Dustin Getzthat's where we're at right now#2022-12-1220:04Dustin Getzwe already have the discrete version, it works fine#2022-12-1220:05Ben SlessWith feedback loops, too?#2022-12-1220:06Dustin Getzyeah we cycle by reactive assignment, Leo did the initial work here
(defmacro with-cycle [[s i] & body]
`(let [a# (atom ~i) ~s (p/watch a#)]
(reset! a# (do #2022-12-1220:06Dustin Getzthat's well defined in missionary master (it may always have been, i dont recall)#2022-12-1220:08Dustin Getzthis runs, it's a datomic backed webform and models transactions as a closed loop over client/server; it has load states, queries, edits, everything#2022-12-1220:09Ben SlessI don't know if the entire system could be modeled in continuous time because the communication between client and server is definitely discrete, but a local context could perhaps be reasoned about in continuous domain#2022-12-1220:10Dustin GetzLeo solved that, the network runtime is discrete but the userland model (photon functions) is signals#2022-12-1220:11Dustin Getzour reactive p/fns ("photon signal functions") can be thought of as including a reader/writer monad for network events (I think that's a fair summary), obviously the internals are discrete#2022-12-1220:12Dustin Getzthe network stuff is entirely abstracted, it doesn't leak to userland at all except at the entrypoint#2022-12-1220:12Ben SlessI think I'm out of my depth here because I'm not really familiar with photon#2022-12-1220:15Dustin Getz"differentiable signals" your definition of this is mcsherry?#2022-12-1220:16Ben SlessFrom learning Signal Processing and lots of related material in uni#2022-12-1220:19Ben Slessin discrete time it maps to modelling every function with deltas, for example (merge A+da B+db) = (merge A B + merge da B + merge A db + merge da db)#2022-12-1220:19Ben SlessThen the differential is just the difference#2022-12-1220:20Dustin Getzgot it, i think what we have in the gist above is that#2022-12-1220:21Dustin Getzdb = Σ [t=0..T] App(t)Δt
App(t) is the App fn, it basically returns datomic txn values in response to user interaction. i took some liberties with control flow to get good syntax#2022-12-1220:22Dustin Getzmaybe you mean something different, i'm not sure#2022-12-1220:22Dustin Getz(datomic txn values are literal database deltas)#2022-12-1220:23Ben SlessYeah, I meant it for all data, not just the db, but maybe you don't need that
I should go sleep, I'll be more coherent tomorrow#2022-12-1220:36Dustin Getzthe datomic types are probably making it hard to reason about for us both, if the "database" was just a shared global counter and "deltas" were just [1 1 1 1 1] i think it would be more clear#2022-12-1220:39Dustin Getzthe dom syntax is macros also, the macros emit simple effects like dom/appendChild dom/setText dom/removeChild (supervised by missionary of course for lifecycle guarantees) - that might count as programming with derivatives#2022-12-1220:55Dustin Getzanother problem is that photon has full clojure interop, so even if photon DAGs/circuits could be like symbolically differentiable (?) in the equational sense, and maybe they are?, the clojure/java/js interop is not#2022-12-1218:50Ben SlessBut since you're dealing with partially ordered signals, have you looked at timely dataflow and differential datalog?#2022-12-1313:00Yab MasWhat is the situation with respect to targeting nodejs for the backend? From what I understand it should be possible, but all demo's seem to run on the jvm so I guess thats what's currently supported. Is it on your roadmap or is this something that will be left to the community to develop? If so, can you give an estimate of the work that needs to be done to support it?#2022-12-1314:58Dustin Getzlet me talk to the team#2022-12-1523:36Dustin Getzwe do not support node today but there are no serious barriers. some design work needed (for example we are resolving vars with eval on the backend; the frontend is compiled) and some code will need to be made more portable. i’m not sure if the work will need to be done by us, we would need to do a proper discovery. i would guess yes. #2022-12-1523:37Dustin Getzdo you have a specific business use case in mind? #2022-12-1612:34Yab MasOk, thanks for the reply. We'll be evaluating the adaption of photon over the next few months. Most applications currently target node for the backend, hence the question. We're just exploring options for now. We do consider moving (back) to the jvm, based on more factors than just photon. But making photon node compatible could also be considered if the amount and difficulty of the work seems doable.#2022-12-2111:04mitchelkuijpers(Ignore i fixed it, it was a user error)
I am probably missing something obvious but when I startup photon with:
clj -A:dev -X user/main
And then
(user/main)
I keep getting:#2022-12-2111:13mitchelkuijpersI am trying to run from master btw#2022-12-2111:20mitchelkuijpersMy bad I changed the ring-jetty-9 adapter to a wrong version... it works now!#2022-12-2618:07noonianHello and happy holidays!
I'm seeing changes in behavior that was previously working and wondering if I've found a regression. I haven't pinpointed a super narrow example, but what I'm seeing is when I pull data from a map via keyword (i.e. (:some-prop m)) inside photon dom code, the value from the map is rendered multiple times, both in the desired element and also in it's parent elements within the photon defn (until the next client/server boundary possibly). This does not occur if I pull the value out of the map in a let or for binding and only refer to the value as a symbol in the dom code. I was able to reproduce this in the demo_4_chat_extended demo with the following changes, and the rendered behavior (note username "noonian" is rendered twice, once in the <strong> tag and once in the parent <li>)
Please let me know if this is user error on my part. As always thank you for your work on photon and there is no urgency on my end for a fix.#2022-12-2619:13Dustin Getzpretty sure this is us i will take a look in a few days, team is on vacation and i’m doing taxes#2022-12-2619:19Dustin Getzalso are you using Photon for anything? i need to know if people are using it so i can make judgment calls about breaking changes as well as communicate idiom changes. i try to stay in touch with everyone who is making something#2022-12-2619:39noonianI have a project I've been using for experiments, but I don't know what the final incarnation will look like and I have not been very consistent working on it (just dusted it off after three weeks without touching it). I plan to deploy it and experiment with some operational things like horizontal scaling with redis for shared state etc. and running stateful services on kubernetes.
I'm very excited about Photon but I'm still trying to get a sense of it's sweet spot for target applications so I can make something that takes advantage of it's super powers. I've been trying to follow the src-docs and your scratch namespaces to get a sense of how idioms are developing. I'm just doing this on my own and no one is paying me so I can't commit to making something real. I'd love updates about breaking changes etc. but I understand it's still in alpha and you have to manage your time#2022-12-2708:53xificurCThanks for reporting this!
TLDR: This is most likely from 5ba144d094f9ebb4943e88aa4263db1ba933c940 (Dec 7th). We'll check in the new year if we can revert it safely.
There are 2 competing forces at play here. Clojurists are used to hiccup, so if they wrote
[:div [:strong (:name e)]]
They'd like that to translate to
(dom/div (dom/strong (:name e)))
. But dom/div is a function (macro), not a data structure. Same for dom/strong. So in (dom/strong (:name e)) the dom/strong macro's first argument will be a runtime string. If we want that to work we have to interpret stringy runtime values as text you want to put into the DOM. So the above code is rewritten to
(dom/div (dom/strong (dom/text (:name e))))
. dom/text puts the text into the DOM and returns nil . If it returned the string then dom/div would see it as the return value of dom/strong and re-interpret it, which would result in you seeing the string twice in the DOM.
Now consider this pseudocode:
(let [name (dom/input (InputValue.))]
(println "Your name is" name)
In this case you want to use the return value of dom/input but not for DOM text injection.
The change from Dec 7th wants to make both of these cases work, but as you noticed this isn't completely possible. We're still iterating on a good API.#2022-12-2714:27nivekuilhave you seen how fulcro dealt with this? scroll up to the NOTE bit from https://book.fulcrologic.com/#_props#2022-12-2717:10Dustin Getzreact controls don’t return events, in react events are handled through callbacks. in FRP, especially in low level Photon code like implementing fancy controls, we emit events in a pure functional style via the return channel. this difference leads to a syntax collision where string values are ambiguous - is a string an FRP “event” or reactive value to propagate up?or is it dom body innerText.#2022-12-2717:12Dustin Getzi have a proposed style that makes static props explicitly static (no need for runtime check) as well as disambiguates the body text, but we haven’t discussed it as a team yet#2022-12-3014:48Dustin GetzOk I took a look at https://github.com/hyperfiddle/photon/commit/5ba144d094f9ebb4943e88aa4263db1ba933c940 , i think it should be reverted but I don't dare do it until we're back from vacation due to breaking whatever thing wanted the opposite behavior#2022-12-3014:49Dustin GetzYou can revert it locally if you want @U052PH695#2022-12-3014:56Dustin GetzIt appears there's an interaction with another change, I was able to reproduce it until this commit from this morning https://github.com/hyperfiddle/photon/commit/6c407cf0b5e96b5475d8c2a6788996ef10e2d2f0 which causes the reproduction to no longer work#2022-12-3014:57Dustin GetzI propose you just pull latest, i think it's sorted. We are revisiting all this stuff soon anyway, none of us are happy with the photon-dom1 impl#2023-01-0217:52denikHaving a race condition in photon between client/server state and can’t seem to figure out how to use Unglitch properly#2023-01-0219:09Dustin GetzThe first glitch, what is the symptom you seeing? The picklist resultset changes and therefore the record for chosen index shifts?#2023-01-0219:09denikyes, suggestions updates before the on-pick. runs#2023-01-0219:10Dustin Getzah, the on-pick runs only once (too late), not twice?#2023-01-0219:10denikchecking#2023-01-0219:11Dustin Getzrecently-searched-ents is an atom on the server?#2023-01-0219:11denikyes temporarily#2023-01-0219:11denikthere is a p/watch on it but in a different view#2023-01-0219:14denikit runs once, even with (reset! !input "") uncommented#2023-01-0219:17denikbut it seems that because (reset! !input "") runs, input updates, suggestions are recomputed for input being "" and then those suggestions run in the p/server expression#2023-01-0219:17Dustin Getzi understand#2023-01-0219:17denikI tried binding a nil return value of the p/server block into the client to sequentialize execution but it did not help#2023-01-0219:18Dustin Getzi almost have it, 1 min#2023-01-0219:19Dustin GetzWhat happens if you do this:#2023-01-0219:19Dustin Getz#2023-01-0219:20denikno bueno#2023-01-0219:20denikwhen (reset! !input "") is uncommented it does not work#2023-01-0219:21denikimportant to note that this bug only happens when input is not= ""#2023-01-0219:25Dustin Getztry moving it inside the p/server#2023-01-0219:25Dustin Getzone line down#2023-01-0219:26denikno success#2023-01-0219:27Dustin GetzIs there a way I can run this?#2023-01-0219:27denikit would take a bit of setting up#2023-01-0219:28Dustin Getz(when show-suggestions?
(suggestions-fn. (p/Unglitch. input)))#2023-01-0219:30denikthat also did not work#2023-01-0219:30Dustin GetzAnother option is
(when (p/server
(let [picked (get suggestions idx)]
(if picked
(on-pick. picked)
#_(on-create. input))
true))
(reset! !input ""))
but this will lag your UI#2023-01-0219:30denikI tried this yesterday and it didn’t work but let me try again#2023-01-0219:31Dustin Getzpicked (get (p/Unglitch. suggestions) idx) as well, the goal is to slow down the suggestions query as it is updating too fast#2023-01-0219:34Dustin Getzi'll try again later today, i need to eat lunch#2023-01-0219:35denikokay.#2023-01-0219:35denikwith the when it runs twice#2023-01-0219:35denikhowever, it seems only the second execution of on-pick. succeeds#2023-01-0219:36Dustin Getzi dont understand#2023-01-0219:36denikwith this is runs twice https://clojurians.slack.com/archives/C7Q9GSHFV/p1672687807671769?thread_ts=1672681969.404549&cid=C7Q9GSHFV#2023-01-0219:37denikaccording to logs in the p/server block#2023-01-0219:37Dustin Getzshow me your code with logs#2023-01-0219:39denik(do
(.preventDefault e)
(println :running-client)
(when (p/server
(println :running-server (count suggestions))
(let [picked (get suggestions idx)]
(if picked
(on-pick. picked)
#_(on-create. input))
true))
;; FIXME can't run this due to glitch bug
;; p/Unglitch. is not working
(reset! !input ""))
)
;; =>
;; :running-server 2
;; :running-server 20
#2023-01-0219:39denikif this doesn’t help I’ll try to modify the Typeahead view this is based on to fail in the same way#2023-01-0219:40Dustin Getzyeah that's where i will start when i try again#2023-01-0219:45Dustin Getz(do
(.preventDefault e)
(when (p/Unglitch. input)
(p/server
(let [picked (get suggestions idx)]
(if picked
(on-pick. picked)
#_(on-create. input))
true)))
(reset! !input ""))#2023-01-0219:47denikalso not working unfortunately#2023-01-0219:47denikit’s not urgent#2023-01-0219:47denikI’ll work on runnable repro code soon#2023-01-0219:50Dustin Getzok#2023-01-0310:16xificurCI think this isn't related to the distribution glitch. on-pick and on-create
might not finish running because you reset! and cause new reactions. What
you're looking for is to run the reset! after the photon function runs.
If a photon function transfers it will throw a Pending exception first. We can
use a when or case to wait for exceptions to resolve before running
subsequent code. Try this version:
(case (if picked (on-pick. picked) (on-create. input))
(p/client (reset! !input "")))
I find this unreadable since it doesn't capture the intent so I would use a
macro like:
(defmacro sequential [& body] (when (seq body) (reduce (fn [ac nx] `(case ~nx ~ac)) (reverse body))))
to write:
(sequential
(if picked (on-pick. picked) (on-create. input))
(p/client (reset! !input "")))
It's the same code but more intentive and the macro allows chaining further without nesting, e.g.
(case (foo)
(case (bar)
(case (baz)
(quux))))
;; vs.
(sequential (foo) (bar) (baz) (quux))
Please consider this a hack more than a solution. Our goal is to find patterns
where code like this is not necessary.
If this still doesn't resolve your issue it'd be great to have a snippet we can run locally 😉#2023-01-0314:44Dustin Getzthe problem with "sequential" is it delays the UI from updating locally until after the server effect finishes#2023-01-0314:48Dustin Getzthe problem iiuc is when we press enter, we race the clearing of the input (and subsequent refresh of the options due to the input clearing) - we race that with the change handler which is intended to be a discrete side effect that should happen only once. The goal is to make the keyboard-event DAG unmount before the change handler runs again, and unglitch is a way to delay the query reactions in response to clearing the input until after that keyboard-event DAG unmounts.#2023-01-0314:49Dustin GetzPerhaps calling it the d-glitch is not accurate but i think unglitch solves it for now#2023-01-0314:49Dustin Getzobviously i am vulnerable to having missed something because my suggestions didn't work (including the suggestion to sequence the effect and accept a lagged UI)#2023-01-0314:51Dustin Getzwe will land a real solution to this along with photon-dom2 and photon-ui3 and the photon runtime fix to the d-glitch, which will all likely come together in about a month#2023-01-0321:04denik@U09K620SG here’s simplified repro of the bug#2023-01-0409:11xificurCthanks! I just tried this locally and the temporary fix I posted solves your issue#2023-01-0409:15xificurCthis makes sure !input is reset only after the callback has run, solving the race.
We'll be working on new ui components, including new typeaheads, in the coming weeks#2023-01-0418:56denikthanks @U09FL65DK that fixed it :)#2023-01-0419:53xificurCglad to help!#2023-01-0917:52deniki fixed it by putting suggestions into an atom and derefing it instead of using suggestions directly in the callback#2023-01-0917:55Dustin Getzthat was a great idea#2023-01-0217:53deniksee the two fixme’s in the code:#2023-01-0219:00denik#2023-01-0422:03denikis there a way to get more detailed information on serialization errors? I’m getting a lot of
DEBUG : Unserializable reference transfer:
which makes things hard to debug. For example, is the Failure what failed to serialize here or what?#2023-01-0422:08Dustin GetzFailure is an internal type that wraps exceptions, we do transfer exceptions across peers, i forget exactly how they are serialized, some userland exception here may be failing to serialize#2023-01-0422:09Dustin GetzI agree this is not very helpful#2023-01-0422:17denikthe strange thing is that the same code with the same arguments runs fine in the clojure process but fails executing inside photon#2023-01-0422:18denikand that’s regardless of the return value#2023-01-0422:43Dustin Getzwhat is the code?#2023-01-0422:44Dustin Getzyou can also track down that error string and change the str to pr-str, or deep def it all the way down there or something so you can get to it at the repl#2023-01-0422:44Dustin Getzonce you know where the exception is originating, you can try catch tightly before any transfers and pr-str#2023-01-0422:46Dustin Getzunfortunately you cannot deep def in photon as due to a legacy mistake, ‘def’ is a low level special form in photon that needs to be given a different name so that deep def can work#2023-01-0422:51denikit seems to be due to a var that is being closed over by another function and later redefined that then isn’t available in photon#2023-01-0422:51denikagain hard to share code. I’ll try fixing it and share simplified repro code if needed#2023-01-0517:23Dustin Getz@denik websocket reconnection is pushed and tested on mobile safari#2023-01-0517:24Dustin GetzNote that if the server has redeployed and the client connects to a newer server version, the reconnection will occur despite the code version mismatch, and this is undefined behavior#2023-01-0517:24Dustin GetzWe opened a ticket to deal with this, likely by validating a hash of the program or something like that and asking the user to refresh the page to get the new version. Transparently upgrading through continuous deployments is possible long term but not a priority#2023-01-0517:54denikthank you!! will test asap!#2023-01-0518:07Dustin Getzlet me know, i wasn't able to get my phone to sleep to repro your actual issue (for me on mobile safari the websocket stayed alive no matter what i did to my phone, just throttled)#2023-01-0519:18denikwill the reconnect survive client/server state#2023-01-0519:18denikI think that was why yam blew up in the past#2023-01-0519:47Dustin Getzthe app will reboot. We did implement resync but in testing found a design flaw in Photon, we intend to address it in the next photon iteration (starting now)#2023-01-0519:48Dustin Getzyou'll need to put your state in durable atoms, so backed by localstorage, url, database etc#2023-01-0520:46denikonce there’s an error this change seems to land in a loop where the frontend keeps restarting and retriggering the error#2023-01-0921:21Dustin GetzLeo pushed fixes to this#2023-01-0921:21Dustin Getzmaybe wait a few days for us to shake out any more bugs - up to you#2023-01-0921:56denikokay, I can work with it for now#2023-01-0921:56denikhowever, the frontend now crashes when the backend errors#2023-01-0921:59denikI know what causes the error in this case. the server is actually fine. it’s one p/server expression calling a function in a p/defn view that errors on certain inputs. it seems that view could error in isolation without blowing up the rest of the stack.#2023-01-0922:04Dustin Getzok let me think about that#2023-01-0922:04Dustin Getzdo you have a try/catch everything at the entry point?#2023-01-0922:05denikyes
(p/defn Main []
(try
(let [!path (m/mbx)
route (decode-path (router/path !path) hf/read-edn-str)]
(binding [router/Link (router/->Link. !path encode-path)
dom/node (dom/by-id "root")]
(p/server
(views/App. route))))
(catch Pending _)
(catch Cancelled e (throw e))
(catch :default err
(js/console.error (str (ex-message err) "\n\n" (dbg/stack-trace p/trace)) err))))
(def ^:export main
#?(:cljs
(p/boot
(Main.))))#2023-01-0922:05denikis throwing cancelled a problem?#2023-01-0922:06Dustin Getzyou’re throwing cancelled ?#2023-01-0922:06Dustin Getzoh here#2023-01-0922:06denikI basically copied https://github.com/hyperfiddle/photon/blob/571e6125f402d19f46ed875ef4497f1045ba2029/src-dev/user_main.cljc#2023-01-0922:06denikwhat’s the most sensible thing to do?#2023-01-0922:08Dustin Getzi need to follow up with the team about that line#2023-01-0922:11Dustin GetzTo repro the issue, i tried
(ui/button {::ui/click-event (p/fn [e]
(p/server
((identity nil)) ; crash
(swap! !x not)))}
"toggle client/server")
but it doesn't disconnect#2023-01-0922:12Dustin Getzwhat exception is being thrown in the view that is crashing the server reactor?#2023-01-0922:13denikit’s the other way around: the server is throwing an exception that crashes the client#2023-01-0922:14denikit’s just a nullpointer exception#2023-01-0922:14Dustin Getzin my example i have a NPE on click via (p/server ((identity nil)))#2023-01-0922:14denikI wonder if the issue is that transit can’t serialize the exception?#2023-01-0922:14Dustin Getzi think we send a dummy RemoteError#2023-01-0922:17Dustin GetzI'm not sure what to try next to reproduce#2023-01-0922:32denikthis does repro for me https://clojurians.slack.com/archives/C7Q9GSHFV/p1673302314951079?thread_ts=1672939429.797809&cid=C7Q9GSHFV#2023-01-0922:33denikand causes a reactor failure#2023-01-0922:57Dustin GetzFollowing up in public after DMs,
this blows up: (comment (encode (Failure. (Remote.)))) at REPL from (tests
(+ 5 5) := 20)
❌ user:33
in (+ 5 5)
10
:≠
20
It seems if I copy/paste the same reporter I don't get the above anymore, because :expected and :actual are different?#2023-01-2407:55xificurCHi didibus, could you provide a bit more information please? What output are you getting right now? What setup are you using?#2023-01-2618:47didibusRight now I get the default output, it just says:
actual:
expected:#2023-01-2618:48didibusWhat I want is the code under test, then the inequality#2023-01-2619:49xificurCThanks. On master rcf if I start a REPL for your code I get
expected: (= 20 (+ 5 5))
actual: (not (= 20 10))
What is missing from this output for you?#2023-01-2620:26didibusI need (+ 5 5) seperate from the RCF code, the expected code is not (= 20 (+ 5 5)) but only (+ 5 5)
And on the actual, I want it like it was:
20
!=
10
#2023-01-2620:27didibusSo I'd want something like:
(+ 5 5) failed test at user:33
10
!=
20#2023-01-2620:36didibusIt's not that something is missing, but the way it is outputted it takes me longer to parse, which slows me down. I'd like a clearer output so it's faster for me to figure out what code failed what test and how it failed.#2023-01-2620:58xificurCI see.
The output you see matches a clojure.test report, we've been making changes to be more compatible with the ecosystem around it to gain more leverage.
This is very crude but shows that the reporter is a standard clojure.test reporter. The output of this is
Failed: (+ 5 5)
10
!=
20
The map m also has :file and :line metadata in my REPL#2023-01-2623:05didibusAre these nth reliable? Or only works for this example?
Ya, I saw it's changed to be the standard clojure.test output. But I find that a bit hard to parse on the REPL when I run my tests.
I couldn't find a way to extract reliably from the map what was my code from the code added by Rcf for the check.
I'll try what you pasted and see if it's reliable.#2023-01-2517:55Ben SlessTried to take RCF for a spin today and I had some issues with its behavior when generating a test. I wrote a failing case, lein test didn't report it as a failure, and when I used cider-run-loaded-tests it reported it ran it as a test function but it didn't fail
From reading the documentation and skimming the source I'd expect it to behave like any other test I defined with deftest assuming *generate-tests* was true#2023-01-2518:03Dustin Getzit sounds like a configuration or classpath issue out of band of RCF to me, do we know how lein test detects tests and what cider-run-loaded-tests does? they may have restrictions on file naming conventions and folder placement that RCF ignores#2023-01-2518:03Dustin GetzCan you confirm if this entrypoint style works:
{:aliases {:test {:jvm-opts ["-Dhyperfiddle.rcf.generate-tests=true"]}}}
clj -M:test -e "(require 'example)(clojure.test/run-tests 'example)"#2023-01-2518:08Ben SlessI'll try to set up several examples with various degrees of complexity and see when it breaks#2023-01-2906:14didibusSeems RCF messes up with the &env of macros?
{i {:op :binding, :name i, :init {:op :invoke, :form (. clojure.lang.Numbers clojure.core/long_array 1 0), :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :fn {:op :symbol, :local? false, :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :form ., :ns nil, :name .}, :args [{:op :symbol, :local? false, :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :form clojure.lang.Numbers, :ns nil, :name clojure.lang.Numbers} {:op :symbol, :local? false, :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :form clojure.core/long_array, :ns clojure.core, :name long_array} {:op :const, :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :form 1} {:op :const, :env {:locals {}, :namespaces {}, :ns com.xadecimal.procedural}, :form 0}], :children [:fn :args]}, :form i, :local :let, :children [:init]}}
I have a macro that uses &env, and I can't test it using RCF, because the macro no longer receives normal LocalBindings.#2023-01-2909:51Geoffrey GaillardRCF has its own analyzer, env and macroexpander. However, a userland macro should be called with a regular &env value. I'll create an issue.#2023-01-2909:56Geoffrey Gaillardhttps://github.com/hyperfiddle/rcf/issues/74#2023-01-2917:03Dustin Getz@U0K064KQV can I see your macro? I am considering possible solutions and want to understand better the pattern#2023-01-2917:48didibusIt's a non trivial macro. What I do is that I attach metadata to locals I create in one pass. And use the &env to inspect that metadata on another pass.
I use that metadata to learn about the type hints and also to distinguish the locals my macro created versus the ones the user created.#2023-01-2917:53didibusSo &env is used to get all current locals, their primitive type, and their attached metadata.#2023-02-0123:31Dustin Getzdon't pull master, we're landing breaking changes#2023-02-0516:27Dustin GetzI am looking for a keyword argument parser library that can parse this with spec or malli, does it already exist?#2023-02-0805:31Ben SlessSpec's keys*#2023-02-0517:11Noah BogartCan you be explicit about what you want? I don’t quite know based on those limited samples#2023-02-1015:01denikmade some sparklines using photon (server side http calls and svg generation)#2023-02-1221:03Dustin GetzPhoton is now Electric Clojure, and the repo is now public! https://github.com/hyperfiddle/electric#2023-02-1309:29simongrayI like the new name#2023-02-1221:16Dustin GetzThere are minor breaking changes; I will do a release notes this week#2023-02-1305:59tatuttried to todo example, feels like magic#2023-02-1306:01tatutthe readme mentions network planner improvements, is that related to the chattiness of the WS connection? it looks that currently clicking a checkbox in a todo item sends ~40 messages and ~6kb of data. It works very well locally.#2023-02-1309:45xificurCYes, it is related. Note that we routinely run the demos and other apps under high latency and the ~40 messages don't lag as much as you might expect#2023-02-1311:59borkdudeI posted it on HN, it's on the first page now 😅#2023-02-1312:00borkdudeSince there was no "About" in the electric repo, I chose the title "Electric Clojure: unified fullstack application compiler#2023-02-1313:12tatutit looks like the dom element macro should work with custom elements, so you could use electric to drive some js library that is wrapepd in webcomponent custom elements#2023-02-1409:30dumratI saw it mentioned that there are Zoom onboardings#2023-02-1415:03Dustin GetzHey, yes! Will DM#2023-02-1409:30dumratAny details?#2023-02-1410:02Mario TrostHi 👋
I'm just now reading through the Hacker News discussions and saw @dustingetz mention this:
> Mechanically, Electric is comparable to Solid.js except the reactive engine is general purpose, not coupled to DOM rendering, which is a special case of incremental view maintenance.
Just a couple hours ago I saw this tweet about the new Solid.js dev tools: https://twitter.com/thetarnav/status/1625177986197778434?s=20
(and asked myself for the first time if there is any similarilty there or if you just both use the same words "signal" and "reactivity" 🙂 )
Thought I'd share this here in case you missed and congrats on the release!#2023-02-1414:19Dustin GetzThanks! The technical approaches are similar, we've experimented with DAG visualizers in the past, that is likely how an Electric debugger should work#2023-02-1507:58Mario TrostThat (and everything else) sounds very promising, looking forward to all that's to come#2023-02-1411:46OmarQuestions:
Will it play nice with a react native app? Looking at the reagent demo it looks like this would be possible.
Also nearly all the example have db transact/query code mixed with UI in the same file much like a PHP file. It wouldn't be a problem to extract that out into a separate namespace to reuse and expose to a traditional REST api for mobile apps?
I'm not familiar with the computer sciencey magic that's going on to make all this possible so I'm not capable of assessing the tradeoffs vs a traditional backend and mobile app/web frontend codebase.
I really love what I see and if these concerns of mine are non issues I'll probably spin up a pet project pretty soon and see how all of this plays together.
There's probably a lot of low hanging fruit with regards to documentation/demos for unsophisticated fullstack people like me because the demos make it look a lot easier to grasp than the introduction does. I don't have a computer science background but have immense experience building web apps. I'm typically one to let tech mature before giving it any attention but what I see is compelling.#2023-02-1411:56Geoffrey Gaillard> Will it play nice with a react native app?
Sure, the reagent demo could be adapted to react native.
> Also nearly all the example have db transact/query code mixed with UI in the same file much like a PHP file. It wouldn’t be a problem to extract that out into a separate namespace to reuse and expose to a traditional REST api for mobile apps?
You can call any clojure code from electric clojure. Just move the query in some shared namespace, so you can call it from both your REST endpoint and your electric Clojure program.
> I’m not familiar with the computer sciencey magic that’s going on
No magic 🙂 have a look at Missionary to see what’s under the hood (https://github.com/leonoel/missionary)
> if these concerns of mine are non issues
All concerns are valid. Thank you for sharing them.
Feedback is welcome and helps us a lot. Don’t hesitate to share what you are trying to build and the difficulties you are facing, so we can improve.
Thank you for the kind words 🙂#2023-02-1414:37OmarI think once it has a bit more reagent/hiccup support I could see myself plugging this into an existing project I'm working on for sections I want "live." I could see using electric as my main websocket backend, running alongside my existing backend instead of going through the hassle of using sente and a mix of re-frame events/subs.#2023-02-1414:42Omar... and you just dropped the reagent demo as I was typing this 😉 Will see what I can do with it.#2023-02-1411:56Geoffrey Gaillard> Will it play nice with a react native app?
Sure, the reagent demo could be adapted to react native.
> Also nearly all the example have db transact/query code mixed with UI in the same file much like a PHP file. It wouldn’t be a problem to extract that out into a separate namespace to reuse and expose to a traditional REST api for mobile apps?
You can call any clojure code from electric clojure. Just move the query in some shared namespace, so you can call it from both your REST endpoint and your electric Clojure program.
> I’m not familiar with the computer sciencey magic that’s going on
No magic 🙂 have a look at Missionary to see what’s under the hood (https://github.com/leonoel/missionary)
> if these concerns of mine are non issues
All concerns are valid. Thank you for sharing them.
Feedback is welcome and helps us a lot. Don’t hesitate to share what you are trying to build and the difficulties you are facing, so we can improve.
Thank you for the kind words 🙂#2023-02-1414:37Geoffrey GaillardThis demo is now available in the Electric Clojure repo. https://github.com/hyperfiddle/electric/blob/fee5c441ad809def70b17ce4f7b7c537f3a37a09/src-docs/user/demo_reagent_interop.cljc.#2023-02-1500:44OmarIs it possible to pass in an e/fn block or something equivalent to work within reagent? For example sending data to server from an existing form input.
I tried and I'm getting this error: {message: 'Invalid p/fn in Clojure code block (use from Electric code only)'#2023-02-1501:43Geoffrey GaillardAs of today this is not possible.
As a workaround, you can pass a callback or an atom from electric clojure to your reagent component. You can then use e/watch to react to changes on the atom.#2023-02-1421:22dangercoder👋 Would be thankful if someone could point me in the right direction for implementing auth. I’ve read some of the source code and couldn’t figure out a way to extend e.g. the ring handlers.
https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_jetty_server.clj#L15#2023-02-1421:31Dustin GetzWe will take a look at making a simpler demo, what specific requirements are you looking for?#2023-02-1421:31Dustin GetzIf you send me a gist of what you have that would be helpful#2023-02-1421:44dangercoderMy requirement is:
A server-side route that redirects the user to Steam for login. It needs to go via the server so I can store some data in a session cookie.
It's pretty simple in ring, just two endpoints. One for logging in and one for the auth callback.
This requires being able to specify routes, middleware and handlers. Not sure if it's out of scope for this project, in that case I apologize gratitude-thank-you.#2023-02-1421:45Dustin Getzis there a ring-starter-app of some sort that we can fork, so that the question simplifies to "make Electric work in this repo with auth"#2023-02-1514:01Dustin GetzHey we took a look this morning, we think the electric_jetty_server file has what you need. Can you add me to your repo so I can take a closer look at what you have and understand your specific context?#2023-02-1521:58dangercoderHey I will come back to you on this. I'm currently unable to look into this for a while, got some other things I need to prioritize. I'll come back with an example repo.#2023-02-1521:59Dustin Getzok, no worries#2023-02-1421:40pez👋 I am trying the starter app. It just started and I can do multiplayer grocery-listing. So sweet! I'm trying to grasp what is happening still, my mind is blown. To get a feel I tried to do (def id id) in the TodoItem function. I can evaluate the function and the namespace in both CLJ and CLJS, but it blows up with ”ExceptionInfo: Unable to resolve symbol: id” if I then try to restart the app. Same if I save the file and let shadow-cljs reload things. I'm guessing this is not supposed to work. Still, it confuses me a bit, because (println id) works just fine, both in server and client forms. Can someone fill me in a bit on why I get this error?#2023-02-1421:41Dustin Getzdef is shadowed by Electric due to a legacy bad decision, sorry#2023-02-1421:42Dustin Getzyou can use println#2023-02-1421:42pezI'm a heavy inline def user 😃#2023-02-1421:42Dustin GetzWe need to fix it, it's just a legacy cleanup task#2023-02-1421:43Dustin Getzworkaround:#2023-02-1421:43Dustin Getz(defn ddef [v] (def pez v))#2023-02-1421:43Dustin Getzor even#2023-02-1421:43Dustin Getz((fn [] (def pez v)))#2023-02-1421:44pezThanks! I'll try right away.#2023-02-1421:44Dustin Getzwhat's happening is the (fn []) takes you back into an ordinary Clojure context (so no Electric magic in that region of course)#2023-02-1421:45pezI was about to ask, but didn't think I would be able to get the answer, but this makes sense. 😃#2023-02-1421:47pezSo, from both println and this embedded inline def I am now concluding that the code that goes immediatelly in the e/defn body has an implicit server context?#2023-02-1421:47Dustin Getzit inherits the caller's context by default#2023-02-1421:48pezAh, yes, and it is called from the server I now see.#2023-02-1421:48Dustin Getzwe recommend you adopt the convention of calling your Electric functions from the server for now, this recommendation will be lifted soon#2023-02-1708:35xificurC@U0ETXRFEW inline defs landed on master#2023-02-1421:53pezThe October 22 demo video is among the coolest demo videos I have ever seen, btw. Not only is it mindblowing tech, the whole thing is wrapped in the story I need to begin to have an idea what it is all about and what is brought to the table. And delivered in just the right pace for me and with a clarity that is rare to experience. Fantastic.#2023-02-1421:53Dustin Getzwhich video?#2023-02-1421:56pezThis one: https://www.youtube.com/watch?v=fq4_W4vLA6g Maybe earlier than October. But anyway 😃#2023-02-1421:56Dustin GetzOhhh the hytradboi talk, thank you!#2023-02-1422:30pezEarlier today I was watching this video about how React.js came to be. https://www.youtube.com/watch?v=8pDqJVdNa44 (It actually features David Nolen, who helped with making people understand the greatness of the project.) Very cool. And extra cool that I had watched that ^ video about streaming DAGs yesterday. It felt like I knew something they didn't know, haha.#2023-02-1502:27OmarI've noticed on the starter app and in my own project once the electric server goes down that the UI will disappear. How should I manage this? There a way to display anything when it goes down (perhaps even client data?) and execute something when the connection is re-established?#2023-02-1509:58Geoffrey GaillardAn electric clojure program is split in two parts (client and server) at compile time. Both programs are run on their respective peer. They run at the same time and they always go together. The electric server is not supposed to go down unless there is a network issue or the client gracefully shutdowns (navigating away)
In case of network instability, the client peer will reconnect on itss own and reboot the server side.
Because this is functional programming, rebooting the server side with the client state should just restore the page.
The https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_client.cljs has a connection lifecycle, but it is not exposed to userland at the moment.
Could you tell me more about your use case?#2023-02-1510:07OmarYeah! I think being able to still render something to indicate to the user that the app will be back/lost connection instead of going completely blank, then triggering a notification to let them know it's back. If it just completely goes blank that will just be off putting to most people and they'll navigate away from the site.#2023-02-1510:22Geoffrey GaillardUnderstood. I’ll record this in our issue tracker.
In the meantime, one potential solution to mitigate this would be to use CSS. Let me explain:
• your electric clojure program mounts into some https://github.com/hyperfiddle/electric/blob/fee5c441ad809def70b17ce4f7b7c537f3a37a09/src-dev/user_main.cljc#L96 (similar to ReactDOM.createRoot(getElementById(…)))
• When the program starts, it will add children dom nodes to the root element
• when the electric program shuts down, all elements added by electric clojure will be removed
• let’s suppose you root element is <div id="my-program-root"></div>
• you can add, just after it <div id="my-fallback-message"> Connection lost. Reconnecting… (or please refresh the page) </div>
• CSS: #my-program-root:empty + #my-fallback-message {display:none;} . The fallback message will be visible only when the program root is empty.
#2023-02-1509:37JHi guys! Thanks for this amazing lib. I was excited since I saw the hydradboi talk. So again congrats!
Now, I play with it and I’m wonder how integrate sql backend. I test with this code:
(e/defn AddPokemon [!pokemons]
(e/client
(InputSubmit. (e/fn [v]
(e/server
(e/discard
(swap! !pokemons conj (sql/insert-pokemon pg-conn {:id (random-uuid) :display_name v}))))))))
(e/defn App []
(e/server
(binding [pg-conn (sql/create-connection)]
(let [!pokemons (atom (sql/list-pokemon pg-conn))
pokemons (e/watch !pokemons)]
(e/client
(dom/h1 (dom/text "Pokemons"))
(AddPokemon. !pokemons)
(dom/div
(e/for-by :pokemon/id [{:keys [pokemon/display_name]} (e/server pokemons)]
(e/client
(dom/p (dom/text display_name))))))))))
I get the InputSubmit button of the todo demo and I wanted to refresh the list of pokemons when a new pokemon is added to the database. But my code is incorrect because I passe !pokemons server atom directly into the AddPokemon component. How can I achieve this without create a global ref to the !pokemons atoms?#2023-02-1510:07Geoffrey GaillardIf I understand correctly, it looks like you are trying to manually maintain a list of pokemons in the !pokemons atom.
You initialize the atom with a query, then you manually add new pokemons to it.
You cannot ensure !pokemons will stay in sync with the database. I’ll get back to you with a code snippet#2023-02-1510:29xificurCNote that while your code isn't operationally correct (you won't see updates to the list of pokemons done outside from your application) semantically it is OK and will run in the future. !pokemons is passed on to a function running on the client but it never uses it, so the transfer can be optimized away.#2023-02-1516:44JThanks a lot @U2DART3HA#2023-02-1516:59Geoffrey GaillardI’m polishing the SQL snippet a bit more, will share again soon#2023-02-1522:09Dustin GetzHey @UHZPYLPU1 here is the polished snippet - https://gist.github.com/dustingetz/1960436eb4044f65ddfcfce3ee0641b7#2023-02-1606:51JThanks @U09K620SG ! I will take a look! 🙏 #2023-02-1608:04J(e/defn AddPokemon []
(e/client
(InputSubmit. (e/fn [v]
(e/server
(try
(let [x (sql/insert-pokemon pg-conn {:id (random-uuid) :display_name v})]
(swap! !dirty inc) ; success
x)
(catch SQLException e
; handle it, or alternatively let it propagate upward
(e/client (dom/props {:class "error" :style {:background-color "red"}})))))))))
Don’t need to use e/wrap on the sql/insert-pokemon call?#2023-02-1608:08Geoffrey GaillardGood catch. Yes it is needed. All IO blocking operations should be wrapped.#2023-02-1609:08JI have a concern about this:
(binding [pg-conn (sql/create-connection)
!dirty (atom 0)] ; make available anywhere that pg-conn is available
(let [pokemons (e/wrap (sql/list-pokemon pg-conn (e/watch !dirty)))] ; reruns when dirty changes
It’s normal that (e/watch !dirty)is inside the (e/wrap) and inside the (sql/list-pokemon)? :thinking_face:#2023-02-1610:32Geoffrey GaillardYour concern is valid. There’s a mistake in the code.
Sorry for the confusion.
Electric Clojure compiles your code into a graph.
Values propagates through the graph (see below)
• (sql/list-pokemon pg-conn (e/watch !dirty)) is valid.
• (e/wrap … (e/watch !dirty)) is not valid.
e/wrap is a macro taking regular Clojure code.
e/watch is a macro emitting Electric Clojure code.
So e/watch cannot be used in e/wrap.
This is correct:
(let [dirty (e/watch !dirty)]
(e/wrap (sql/list-pokemon pg-conn dirty))
#2023-02-1611:11JThanks @U2DART3HA and @U09K620SG it works!#2023-02-1510:14vlaaadwould be cool to get some sort of protocol defined, so e.g. there is a guide on how to swap compatible servers (is it only a websocket endpoint that matters?), databases (sql/datomic/file on disc), clients (TUI, javafx etc)#2023-02-1510:23xificurCthe client/server messaging protocol happens to be websocket for the web. In our tests we run the client and server peer in the same process, therefore the peers communicate in process. In the end the peers require 1 fn for read and 1 fn for write, so adding new protocols isn't too hard.
We will discuss different databases in the coming weeks, e.g. see the thread above with Postgres. The "hard" part is making the DB notify us there's new data so we should re-run our queries.
We discussed adding a swing demo but didn't find enough time to get to it. The dom library is 300loc, of which 100 is wrapping all the native dom element types in macros, so the whole impl is ~200loc. So adding new UIs also won't be too hard.#2023-02-1514:43Dustin Getz@U47G49KHQ in addition to what @U09FL65DK said,
Jetty is the only server Electric supports today – see electric-jetty-server and electric-jetty-adapter if you want to port them to other servers#2023-02-1514:44Dustin GetzDatabases - we're looking into a Postgres demo, we had Datomic and XTDB as well but they rotted and need to be restored#2023-02-1608:37reedhoI think using with Datalevin is gone be simpler?#2023-02-1608:39reedhoI would like to port adapter for httpkit as an excercise.#2023-02-1510:54simongrayI am wondering how you are going to deal with app hydration in Electric Clojure? Is this something you're working towards...? It would be the holy grail to have an implicitly hydrated full-stack app. i.e. something like a (e/hydrated ...) scope that runs on server or client depending on the hydration state.#2023-02-1511:02Geoffrey GaillardYou mean hydrated as in resuming SSR on the client?#2023-02-1511:10simongrayYes#2023-02-1511:13simongrayBasically, let's say we replace every instance of e/client with a hypothetical e/hydrated . These scopes would run entirely on the server when the initial HTML page is generated and entirely on the client after the initial render in the browser.#2023-02-1511:24Geoffrey GaillardOh yes, definitely.
We have the electric-dom library writing to the browser DOM.
Adapting it to a server-side DOM impl and emitting a stream of static markup could be an exercise for the reader.
Rerunning on the client and correlating SSR generated node ids is doable too.
But it’s not clear to me if hydration would be an actual perf improvement in a model like electric clojure.#2023-02-1511:33tatutexample, if you add a thousand todo items in the sample app, the page load get quite slow, you see a white page for a long time#2023-02-1511:34tatutand then once the websocket receives a huge payload, does rendering begin… similar to SPA apps, I could envision initial SSR showing a big speedup in those cases where you have lots of dynamic data on a page#2023-02-1511:37Dustin Getzi was talking to ryansolid about this, iiuc ssr perf, when the browser is fast, is about eliminating the extra round trip to bootstrap the app from the client. can write more later, also i am not an expert at this#2023-02-1511:39Dustin Getzthe perf hit from your test of adding 1000 todo items is not a “fair” test also, i think it is churning queries and datascript also likely is not responding to cancellation signals, multiple layers are doing bad stuff that needs to be fixed#2023-02-1511:39tatutI might be biased in favor of SSR as my ripley library is all about that#2023-02-1511:41Geoffrey Gaillard"once the websocket receives a huge payload". Electric clojure enables you to avoid this issue entirely. Use e/for on the server to diff your collection, and paginate on the server too. In an production setup, only what's visible to client (in the viewport) should transfer.#2023-02-1511:42tatutyes, agreed… but the browser is super fast at parsing and showing HTML. A thousand todo items wouldn’t break a sweat#2023-02-1511:42tatutso there are tradeoffs to server vs client rendering#2023-02-1511:51Geoffrey GaillardYes definitely. We will need to experiment and measure.#2023-02-1513:11xificurCHow about no hydration, like https://qwik.builder.io/ is doing? Lazy loading the interactive parts as necessary.
The core got mature enough for a public release but we haven't gone through the optimization phase yet. The performance considerations you have in mind will be addressed. Thanks for pushing the limits and thinking of ways to improve!#2023-02-1513:24simongrayTo me it's not just about performance, it's about accessibility. An HTML page where the content is entirely JS-driven is effectively dead in a lot of contexts, i.e. anything that doesn't involve a JS-enabled, modern browser with a direct server connection. Simple SPAs are fine for "apps", but hydrated SPAs can also be used for "websites".#2023-02-1513:25simongrayan entirely JS-driven SPA is ephemeral so it doesn't work for what I would call websites, it is relegated to apps#2023-02-1513:27simongraya hydrated SPA can straddle both kingdoms, i.e you can have a traditional website that can be properly cached and will be a part of history (e.g. in the Wayback Machine) but it can also be highly interactive/dynamic on the client.#2023-02-1513:28simongrayPersonally, having no hydration means I would only ever consider using it for real-time web apps, never for anything website-like. It is still an awesome piece of engineering (seriously awesome) but that definitely limits its scope for me.#2023-02-1513:46xificurCwhich makes sense, we've been focused on building web apps. I logged a ticket in our issue tracker with your comments and we'll think about solutions#2023-02-1513:49simongrayIt's totally fine if this is out of scope, I think I get what you're trying to make here. A boy can dream, though. 😉#2023-02-1514:01xificurCwe're driven by our users' needs so of course we want to make their dreams come true 🙂 I'm not saying this will be done tomorrow of course, but I also don't think it's a harder problem than the ones we've already tackled, so there's no reason this couldn't be done some day#2023-02-1514:05Dustin GetzI want to be clear that Electric Clojure is for ultra-dynamic web applications, not websites. Next.js is excellent at websites and is getting better every month, the new Vercel Cloud Edge Functions stuff seems quite excellent, we will not be able to compete with that#2023-02-1514:09Dustin GetzBasically we need to focus our energies on making Electric world-class in one domain#2023-02-1514:10Dustin GetzCertainly the Electric model can be extended to hybrid approaches, perhaps it's even easy to get a POC working, however matching Vercel's performance numbers is likely not within reach without similar amounts of capital#2023-02-1514:10Dustin GetzVercel raised $313M#2023-02-1514:16Dustin Getz@U11SJ6Q0K I checked the 1000 todo items perf, I changed it to create 1000 records in one transaction to focus the microbenchmark on the rendering. Electric renders the 1000 in 4 seconds. Indeed, that is far too slow and should be optimized. We have a 10k dom elements benchmark internally which also reveals the perf issue here. My intuition is that the dom point effects are not actually the bottleneck. We need to profile, get real numbers, do a proper discovery as to what's actually happening#2023-02-1514:17Dustin Getz@U11SJ6Q0K Do you have any sources that benchmark rendering HTML from document vs rendering HTML from DOM (document.createElement)? There are a lot of confounding factors to eliminate. At the lowest level, both seem to do the same thing: receive a stream of text over network, parse the text, translate the instructions into DOM mutations#2023-02-1514:52tatutI don’t remember anything recent, those might change as browser engines develop#2023-02-1514:53tatutripley just sets server rendered raw HTML as elt.outerHTML = … to bypass JS#2023-02-1514:55Dustin GetzIs it still faster? React.js is 10 years old now, browsers have optimized for it#2023-02-1514:55Dustin Getzalso, event handlers etc#2023-02-1514:55tatutI don’t honestly know#2023-02-1514:55tatutwhat the situation is with current engines#2023-02-1514:56Dustin Getzgotcha me either#2023-02-1518:20OmarPersonally for me SSR isn't important. If speed is the concern you can get around this by doing all you can to optimize your JS build size and getting it on a proper CDN. Before I've also kept my bootstrap data mandatory for an initial render to populate the views embedded as JSON on the html the backend spits out that can be parsed immediately so there isn't an additional API call to fill in your views after the JS executes. Then, when you're intelligent about prefetching things it outperforms any non spa.#2023-02-1511:32pezHot-reloading is a bit less surgical than I am used to. The web page seems to get an empty state and then re-populates. (No idea if this is what is happening, just get that feeling.) Is this something that can be smoothen out in the Electric future?#2023-02-1511:53Geoffrey GaillardYou got it right. Today the electric compiler performs full program analysis. So the whole electric clojure code is recompiled and the client reboots on hot reload to ensure client and server are in sync. Optimizing this is on our roadmap.#2023-02-1514:51Dustin Getz@U0ETXRFEW It's rebooting here: https://github.com/hyperfiddle/electric/blob/fee5c441ad809def70b17ce4f7b7c537f3a37a09/src-dev/user.cljs#L10-L18#2023-02-1514:45Dustin GetzAll: I might do a group onboarding zoom call, please indicate if interested#2023-02-1514:46dumratInterested#2023-02-1613:15jmayaalvinterested!#2023-02-1619:54Dustin GetzFirst group onboarding tomorrow, Friday, 3pm - 5pm ET. Other timezones will be accommodated next week. Here's the deal: you have to have your video camera on. I'm starting with 4 people and seeing how it goes. First 4 people to DM me their email address are in and will get the zoom link.#2023-02-1619:55Dustin Getz@dumrat @U0V0HQWAE @chromalchemy @v @zenflowapp @U9E8C7QRJ @damir.bijuklic @U01SBSXRRH6 @mattias @U03RJ0AMUS1 @sfyire @UHZPYLPU1 @tomisme @jmayaalv#2023-02-1619:56Dustin GetzNext week will do a PT afternoon and a CET afternoon#2023-02-1619:57Dustin GetzReply here or DM me if you need an asia-friendly TZ and we will try to figure it out#2023-02-1623:16PanelHi from Australia#2023-02-1702:42dumratI'm from Singapore#2023-02-1712:02Dustin GetzToday is filled, more next week!#2023-02-2016:47Dustin GetzGroup zoom onboardings this week:
• PT (San Fransisco) this Thurs Feb 23, 3pm - 5pm
• CET (Paris) this Weds Feb 22, 3pm - 5pm
• SGT (Singapore) this Friday Feb 24, 6pm - 8pm
Remember, we do 1:1 onboardings as well which are more effective, please DM me for a 1:1! On our side, the 1:1s help us figure out how to explain this better and discover hidden problems, so we appreciate if you are able to make time for a 1:1. (You can also do both.)
@dumrat @kjw0323 @zenflowapp @chromalchemy @v @damir.bijuklic @mattias @sfyire @tomisme @jmayaalv @sr @jaapmaaskant#2023-02-2016:48Dustin GetzMax 4 people per group, video cameras on, DM me to reserve your spot and I'll email you the calendar event#2023-02-2116:37Dustin GetzTwo slots left for CET onboarding tomorrow afternoon#2023-02-2220:03vincentany slots left? also are these being uploaded anywhere perchance#2023-02-2222:00Dustin Getzyes, i'll add you @v what's your email#2023-02-2310:00dumratPlease add me to SGT (Singapore) this Friday Feb 24, 6pm - 8pm#2023-02-1514:49Dustin GetzWe also do 1:1 onboarding zoom calls to get you up and running much faster, this will save you hours! It also helps us learn about hidden issues and figure out what docs to write, it is not a bother at all. I would personally appreciate it if you would make time to do a zoom onboarding! DM me if interested, please include your social media handle and/or a 1 sentence description of what you might want to do. (This is for my context, it is not a judgment)#2023-02-1515:08Dustin GetzAdam can you give more details about this question? And a link or three
> @adamfrey very excited to hear that. I'm interested if it could be combined with #CVB8K7V50’s model for agnostic platform UI rendering#2023-02-1613:51adamfreyYou or the author of the Membrane library would be more equipped than I am to determine whether there's a fit between the two. But here's some context
From the README:
> Membrane provides all the tools you need to build a user interface. While all the pieces are built to work together, each piece is optional or can be mixed and matched with other implementations. For example, you could use https://github.com/phronmophobic/membrane-re-frame-example (re-frame) and the other layers to reach another platform. Alternatively, you could provide your own ncurses graphics backend and leverage the ui framework and graphics model.
> The three main pieces are:
> 1. A UI framework, membrane.component, that provides state management for GUIs
> 2. A platform agnostic model for graphics and events
> 3. Multiple graphics backends that provide concrete implementations for #2
> For membrane to run on a platform, the only requirements are:
> 1. Drawing implementations for graphical primitives like shapes, text, and images
> 2. An event loop that forwards events (eg. mouse clicks, key presses) to membrane and repaints
The library allows you to describe your UI as a tree datastructure with all relevant rendering info in-band (instead of using HTML and CSS) and then there are different backends for rendering on the desktop via Skia, in the browser via webgl, browser to DOM, terminal apps, and an even experimental renderer for iOS
• https://github.com/phronmophobic/membrane
• https://github.com/phronmophobic/membrane/blob/master/docs/tutorial.md
• https://blog.phronemophobic.com/what-is-a-user-interface.html
• https://blog.phronemophobic.com/ui-model.html
• https://blog.phronemophobic.com/reusable-ui-components.html
• https://github.com/phronmophobic/membrane/blob/master/src/membrane/example/todo.cljc
◦ this example uses all the membrane pieces together, but the library was designed so that you can just use the parts you want to#2023-02-1517:18vincentHi, hyperfiddle looks like the right tool for my next project ^.^#2023-02-1521:19vincentI just cloned the repo and wonder how I should run it. Notice a shadow-cljs so probably run that... Any pointers on how to start this puppy up?#2023-02-1521:21Dustin Getztry readme 🙂#2023-02-1521:22vincentty#2023-02-1521:23vincentAh ok I see the link to the starter app now ^^#2023-02-1521:23Dustin Getzunder Getting Started#2023-02-1521:23Dustin Getzclj -A:dev -X user/main#2023-02-1521:23Dustin Getzwe will have to highlight it better#2023-02-1521:29vincenthot diggity it runs 😄#2023-02-1522:49vincentSo am I correct in assuming that the todo_list.cljc is the server file and also happens to host all the client code x)#2023-02-1600:15Dustin GetzFor others, here is https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#2023-02-1600:17Dustin Getz.cljc is a special Clojure file extension (compare to .clj .cljs) which is processed by both Clojure and ClojureScript. Clojure and ClojureScript are really separate languages with separate (overlapping) toolchains#2023-02-1600:17Dustin GetzThis all has nothing to do with Electric, this is just how Clojure and ClojureScript code sharing works through this .cljc file type#2023-02-1600:18Dustin GetzElectric piggie backs on top of this, but in the end there are two Electric builds - a Server build and a Client build. These two builds both use the same .cljc src as inputs, but the outputs are java .class files and .js files#2023-02-1600:19Dustin GetzThe above kind of simplifies and "white lies" a bit to hopefully help your understanding, it is not strictly accurate but close enough#2023-02-1600:42vincentyeah i like it, having both client and server in one file is cool, i appreciate the explanation#2023-02-1600:43vincentso i am busy brainstorming new ideas now that hyperfiddle greatly simplifies realtime webapp development#2023-02-2016:47Dustin GetzGroup zoom onboardings this week:
• PT (San Fransisco) this Thurs Feb 23, 3pm - 5pm
• CET (Paris) this Weds Feb 22, 3pm - 5pm
• SGT (Singapore) this Friday Feb 24, 6pm - 8pm
Remember, we do 1:1 onboardings as well which are more effective, please DM me for a 1:1! On our side, the 1:1s help us figure out how to explain this better and discover hidden problems, so we appreciate if you are able to make time for a 1:1. (You can also do both.)
@dumrat @kjw0323 @zenflowapp @chromalchemy @v @damir.bijuklic @mattias @sfyire @tomisme @jmayaalv @sr @jaapmaaskant#2023-02-2116:37Dustin GetzTwo slots left for CET onboarding tomorrow afternoon#2023-02-1603:26reedhoHi All, long time no see, congrotulation for the release. ❤️#2023-02-1603:29reedhoI'm more on frontend currently but been watching this since months. Now that it is in public, i want to ask how to use tailwindcss for styling the ui?#2023-02-1607:09Geoffrey GaillardInstall tailwind (follow their readme)
Then add classes to your dom elements:
(dom/div (dom/props {::dom/class ["bg-teal"]}))#2023-02-1607:16JI think you can even use https://github.com/thheller/shadow-css#2023-02-1608:10reedhoLatest i try to configure for its jit feature so e.g. only used classes sent to browser but still not satisfied with it. Hooking with shadow-cljs build pipeline would be awesome. Has some one put ground works on this matter?#2023-02-1608:12Geoffrey GaillardYou want to use its JIT feature to scan your cljc code? To prune unused classes?
What do you mean “only sent to the browser”?
Do you also use tailwind classes on the server?#2023-02-1608:16reedhoYes, i meant for former one, a working good example of tailwind.config.js#2023-02-1608:19reedhoAbout only sent to browser, i mean for development and production it works as exactly as how it works with node react js (e.g. vite) project.#2023-02-1608:28reedhoI think wether this is doable: https://twitter.com/simonswiss/status/1625675040267051010?t=2hHb0DZzXUtzU7tmCp5ftQ&s=09#2023-02-1608:51Geoffrey GaillardUnderstood. It should just work out of the box.
A basic tailwind config would look like:
module.exports = {
content: ["./src/**/*.{clj, cljs, cljc}"]
}
For a prod build you would scan the output javascript file rather than your sources.
While it’s possible, it’s not clear if it’s Electric Clojure’s job to run the tailwind executable, like Remix does.
What do you think?#2023-02-1611:10reedhoI'll retry the things again soon and will show more the findings.#2023-02-1616:47teodorluHi! Electric is interesting. I'm trying to evaluate this from a REPL:
(e/client 123)
;; clojure.lang.ExceptionInfo: Invalid p/client in Clojure code block (use from Electric code only)
;; ...
Is there a quick way I can evaluate electric code from a REPL? (Sorry if this is obvious from some docs I haven't read yet)#2023-02-1616:48Dustin Getzyou need to use RCF#2023-02-1616:50Dustin Getzsee photon_test.cljc#2023-02-1616:51Dustin Getzthe client/server macros require you to have a clojurescript browser repl working, which is tricky (and i actually think we might have broken that recently, i am not sure if our tests pass in the cljs repl right now)#2023-02-1616:52Dustin GetzActually no - there is a special test entrypoint for Electric Clojure that has loopback client/server configuration, photon_test.cljc uses this iirc#2023-02-1616:53teodorluDoes that mean hyperfiddle/rcf provides "magic/helpers/context" required to "choose where to send the expression for evaluation"?
Whereas when I simply evaluate expressions, I don't provide that context?#2023-02-1616:53Dustin GetzElectric is async, so you need async REPL helpers#2023-02-1616:55Dustin Getzhere - https://github.com/hyperfiddle/electric/blob/master/src-docs/user/photon/photon_1_lang.cljc#2023-02-1616:56teodorluAmazing, thanks. That gives me plenty to explore!#2023-02-1616:57Dustin Getz#2023-02-1616:58Dustin Getze/run (p/run) is how you run an Electric expression at the repl, since the program is reactive it will stay running, until you kill the program through the dispose callback#2023-02-1616:59teodorluah, right. I was wondering about that part. Why I needed all the other stuff apart from p/run.#2023-02-1617:00Dustin GetzIt works naked, rcf/tap becomes println outside of (tests)#2023-02-1617:00Dustin Getz#2023-02-1617:01Dustin Getzif you forget to dispose it's mostly harmless in dev#2023-02-1617:02Dustin Getz#2023-02-1617:02Dustin Getznote the println 1 2 3#2023-02-1617:02Dustin Getzin response to the swap!#2023-02-1617:05teodorluso here you are allowed to
(e/def x (e/watch !x))
outside of an async context because x itself becomes an async thing (signal?), but when you actually want to do something with it (print it), you need to provide an async context for that printing with p/run,
(p/run (println x))
?#2023-02-1617:05Dustin Getznothing actually runs until you call e/run#2023-02-1617:06Dustin Getzif you browse into the source, e/run essentially calls the Electric compiler and then boots the reactive program#2023-02-1617:08Dustin Getze/def today doesn't actually do anything; it saves off the quoted program (s-exprs) as metadata for the compiler to deal with later when e/run is called#2023-02-1617:08Dustin Getzthat will change, but helpful for now to understand for your mental model#2023-02-1617:13teodorluok, that makes sense. I'm probably asking lots of stupid questions here, never really worked with async code.
One more question if you don't mind.
(def dispose (e/run (rcf/tap (e/client 123123))))
;; prints 123123
(def dispose (e/run (e/client (js/alert "stuff is happening"))))
;; Syntax error compiling at (src/shadow/user.cljc:13:14).
;; No such namespace: js
Can you help me understand why the second form is failing? I thought since I was able to evaluate client code for the first form, I should be able to for the second form too?#2023-02-1617:19teodorluto try to answer my own question, I'm guessing I'm missing setup. I've been reading the example app, which https://github.com/hyperfiddle/electric-starter-app/blob/b347af77b7c32788acd117777f9d0a1e256d875e/src/user.cljs#L7-L22.#2023-02-1617:35xificurCe/run runs both the server and client parts on the same peer, in this case you're starting it from the clj REPL so on the JVM#2023-02-1617:36teodorluRight. I'm never giving it any information about a clojurescript environment.#2023-02-1617:37teodorluSo I don't really want e/run if I want this example to run.#2023-02-1617:37xificurCWhat you linked runs e/boot, which does the setup#2023-02-1617:38xificurCWell, you can e/run in the cljs repl to try js code#2023-02-1617:39xificurCYou can also build a small entrypoint and let shadow recompile on save. You get similar instant feedback#2023-02-1617:42teodorluI'm using the electric-starter-app example, and I already have recompiles on save.
Is it possible for me to evaluate things in async environment setup by user.cljs? I see that there's a var reactor, can I use that?#2023-02-1617:46xificurCElectric compiles your whole programs, there's no way to "poke holes" into it. Can you give an example of what you'd like to evaluate?#2023-02-1617:47teodorluI'd like to write (js/alert 123) and see that happen in a browser.#2023-02-1617:48teodorluthis is really helpful for me, by the way, I think I'm getting closer to what I'm not understanding.
> Electric compiles your whole programs, there's no way to "poke holes" into it.
I suppose this is what I'm not understanding.#2023-02-1617:49teodorluWait
I have to make some change that gets picked up by the electric recompilation.#2023-02-1617:49teodorluthere's otherwise no way to get information into the frontend.
Does that sound right?#2023-02-1617:50xificurCOK, your example is standard cljs code, so all you need for that to evaluate is a cljs repl#2023-02-1617:51teodorluright.#2023-02-1617:51xificurCIf you add the js/alert into your app then yes, the program will recompile and you'll see the alert#2023-02-1617:54xificurCBy whole programs I mean that electric does full program analysis. Evaluating (e/defn ...) in the REPL doesn't do much, it just saves the sexp for the compiler to analyze it later. So evaling an e/defn in the REPL won't change your running program, you have to save the file for the app to recompile. Does that make sense?#2023-02-1617:55teodorluyes, I think so. I assumed it worked more like a clojure.core/defn, redefining something tangible I could "just touch"#2023-02-1617:58teodorluI think I'm getting "hit by" not really getting when code is being run compile time, (like the e/defn) and when code is executed runtime (like (dom/on "keydown" (e/fn [e] ,,,))). Because inside the keydown event function I can add code that gets run on the frontend.#2023-02-1618:01teodorluReally appreciate you taking the time to explain, Peter and Dustin 🙌#2023-02-1618:17xificurCStandard clojure rules apply otherwise, the electric function will run when you call it. A keydown handler will run when you press a key ;)#2023-02-1618:55Dustin Getz@U3X7174KS see https://github.com/hyperfiddle/electric-starter-app/blob/main/src/user.cljs#L1#2023-02-1618:55Dustin Getz^:dev/always is a shadow-cljs directive to rebuild this file whenever shadow runs#2023-02-1618:56Dustin GetzL8, electric/boot is the compiler#2023-02-1618:58Dustin Getz@U3X7174KS you want to do a zoom call? Any of us are happy to give you a jump start. It also helps us figure out what FAQs people have and know what docs to write next#2023-02-1622:41teodorluI appreciate the offer, but a bit of async Q/A was exactly what I needed right now. Plus, it's soon midnight here in GMT+1!#2023-02-1622:49teodorluOn docs: I think the talks you've given are a great motivation for why electric needs to exist. I want this to succeed.
But then I'm asking myself ... just how can I use this to do something specific, where choosing Electic makes sense? Something like the https://reagent-project.github.io/ or the https://day8.github.io/re-frame/re-frame/, where I'm guided through tasks that get gradually more complex, and there's motivation for why I would want to do the things that appear in the guide. Perhaps that could tackle some of the questions I had above. What does it mean that it's a compiler? That it's all async? How does that give Electric leverage, and what kinds of limitations does it bring?
(my kneejerk reaction right now, I might be wrong!)#2023-02-1623:06Dustin Getzty for the feedback!#2023-02-1623:06Dustin GetzHave you looked at the demos in the main repo?#2023-02-1623:07Dustin Getzin main repo, run clj -A:dev -X user/main , demo code is in https://github.com/hyperfiddle/electric/tree/master/src-docs/user#2023-02-1709:06xificurCThe reagent intro ends with todomvc, you can compare that to https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_5_todomvc.cljc. Disregarding empty lines and diagnostics they come out largely the same LOC. So what's different?
• less moving parts. No useEffect, useRef, ratoms. Larger codebases would also start using useState, useMemo, useContext and whatnot. Electric is just reactive clojure, not much new to learn.
• the electric version has realtime sync. Open 2 tabs and see a change in 1 tab show up in the other. No additional code required. Now think how would you extend the reagent demo to do the same
◦ write the backend part separately
◦ pick a transport protocol
◦ update the reagent code to handle changes flowing in from the backend
▪︎ and this wouldn't be the same pretty code because now you need message handlers that mutate your UI or state.
All of this could easily triple the code size and introduce so much coordination you now need to write, debug and maintain yourself! The toll on your reasoning of the code is huge too, because now you have to keep all these other things in your head when making a change.#2023-02-1715:15teodorluAs of 967f03df, I appear to be getting OutOfMemoryError Java heap space when running clj -A:dev -X user/main. Linux on a 16 GB thinkpad laptop. Happy to provide more details.#2023-02-1715:27teodorluIt appears to be working fine when I set -Xmx1G.
:jvm-opts ["-Xss2m"
"-Xmx1G"
...
]#2023-02-1716:00teodorlu> Have you looked at the demos in the main repo?
I hadn't before you mentioned them. I've spent a bit of time with them now, and it's helpful. I find user.demo-4-webview especially interesting, model, view and controller on one page, all reactive.#2023-02-1716:49Dustin Getz@U3X7174KS I believe that's a compile-time error, -Xss2m fixes it for us (the compiler uses a lot of stack - presumably will be less when we optimize it soon). Please confirm that -Xmx1g is needed as well? That I have not seen before#2023-02-1717:02teodorlu@U09K620SG this is weird - I'm seeing a bit of different behavior.
Observations:
1. When I've already been able to compile the app once, I'm not able to reproduce any compile errors. But when I clj -A:dev -X user/main after a clean git clone or after I've removed untracked files (`git clean -fxd`), I see errors.
2. But after git clean -fxd, I see errors. (I need to run git clean -fxd between each time I test).
a. Neither opt: StackoverflowError.
b. Only -Xss2m: OutOfMemoryError Java heap space
c. Only -Xmx1G: Working
d. Both -Xss2m and -Xmx1G: Working.
So on my system, -Xss2m doesn't appear to have any effect other than giving me different errors, whereas -Xmx1G appears to solve eveything.#2023-02-1717:05Dustin GetzThanks, we will discuss as a team - please keep me posted if you encounter more issues#2023-02-1722:30pezTrying to follow along in this thread about how to use the REPL. Not quite getting it yet, but will keep trying. 😃 Btw, the link to photon_1_lang,clj above doesn't work. It's renamed: https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_1_lang.cljc (For anyone else finding this thread).#2023-02-1722:32Dustin Getz@U0ETXRFEW i have a moment now do u want to zoom?#2023-02-1722:34pezDoes a huddle here work for you? I can zoom too, but just easier with huddle, and anyone can join easier.#2023-02-1722:34Dustin Getzdming#2023-02-1617:57vincentokay we kinda have to rethink our options now with multiplayer canvas as a stock component 😼#2023-02-1617:58vincentI notice in the todo app we have TodoCreate
(e/defn TodoCreate []
(e/client
(InputSubmit. (e/fn [v]
(e/server
(e/discard
(d/transact! !conn [{:task/description v
:task/status :active}])))))))
e/server makes sense and so does e/transact but why e/discard#2023-02-1618:20xificurCThe value from the e/server gets transferred to the client and the return value of the transaction is not serializable. e/discard just returns nil#2023-02-1618:23xificurCInstead of discard you could write (e/server (d/transact! ...) nil)#2023-02-1618:37vincentoh cool, so i kinda understnd -- do you mean serializable like if i wanted to inline the result of the call#2023-02-1619:09Dustin Getz(d/transact! ...) returns like a connection or something as the return value, which can't cross from server to client#2023-02-1619:09Dustin Getzyou'll get a warning in the console "unserializable reference error" - it is harmless#2023-02-1619:10Dustin Getzreturning nil will send nil instead of trying (and failing) to send the reference, resolving the warning#2023-02-1619:13Dustin Getzfor example, (e/client (println (e/server (type 1)))) will try to send a java Class type to the client and print it at the browser console, which will fail to serialize and you'll get an "unserializable reference transfer" warning#2023-02-1619:50vincentokay thanks, didnt mean to melt brain 😅 i appreciate the explanation#2023-02-1619:50vincentwill have to think about it some more#2023-02-1620:04xificurCno problem 🙂 Standard clojure evaluation rules apply, so (e/server 10) returns 10 to the client. In order to "return" it has to cross a wire in standard web app setup, so the value gets serialized (with transit). This means unserializable data is problematic, e.g. (e/server (Object.)). In this case you get a warning and the block will return nil. To signal you don't care about a return value you can use e/discard, which will return nil without the warning.#2023-02-1617:58vincentd/transact*#2023-02-1623:11Dustin Getzin case you missed it, first zoom group onboarding tomorrow 3pm ET - two slots left https://clojurians.slack.com/archives/C7Q9GSHFV/p1676577250341209?thread_ts=1676472352.651429&cid=C7Q9GSHFV#2023-02-1623:25MaxSomeone in my network noted the similarities between Electric Clojure and http://opalang.org/. Was there any inspiration from that project?#2023-02-1707:41Geoffrey GaillardWe designed electric from first principles. We've seen Opa a long time ago IIRC. Electric is fundamentally different. We've seen other languages with client and server markers, but as far as we can tell, they are all doing RPC or message passing. Electric is based on a functional effect system.#2023-02-1708:30xificurClooking at their https://github.com/MLstate/opalang/wiki/Hello%2C-chat they use a Network to broadcast messages between the clients. The clients then use an "on-message" callback to handle incoming messages. This is done in user_updatethat directly manipulates the DOM.
Compare to our https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_4_chat.cljc where we imitate a DB with an atom (could have used e.g. datascript as well) and attach a single impure operation to the inupt - transact on enter. Notice there's 0 ceremony around this, the app is declared and reactivity takes care of fulfilling that declaration.
The execution model is different (messages vs. reactivity) and therefore the code is written differently as well (impure callbacks vs. pure reactive code).#2023-02-1716:51Dustin GetzThere is also http://mozart2.org/mozart-v1/doc-1.4.0/dstutorial/index.html#2023-02-1716:51Dustin Getzhttps://ocsigen.org/home/intro.html#2023-02-1716:51Dustin Getzhttps://en.wikipedia.org/wiki/Multitier_programming has a list of languages#2023-02-1716:52Dustin GetzAFAIK none in that list have the key streaming network planner aspect, which is the critical thing that makes it work, but maybe am missing something, we didn't look too closely#2023-02-1700:13vincentOpa looks awesome if i still practiced necromancy#2023-02-1700:13vincentre: npm deps x)#2023-02-1719:43tatutI tried to make a hiccup like macro, but that interferes with the electric macro (would need to make a macro that wraps the e/defn and processes things before that I guess), and a function that does dom/elementcalls doesn’t seem to work properly in a dynamic case (says hyperfiddle.electric-dom2/node not declared ^:dynamic )#2023-02-1719:43Dustin Getzyeah there are good reasons that we haven't provided hiccup#2023-02-1719:44Dustin Getzthough im not sure what error you're seeing#2023-02-1719:44tatutI can make a snippet of my quick hack experiment#2023-02-1719:45Dustin Getzi can probably help you resolve any issues with dynamics, the boundary between photon dynamics and clojure dynamics has to be explicitly managed currently#2023-02-1719:45Dustin Getzyes please show snippet#2023-02-1719:46tatut#2023-02-1719:46tatutit seems to mostly work tho#2023-02-1719:46Dustin Getzthat's really cute, we have not considered that style#2023-02-1719:46Dustin Getzi like it a lot#2023-02-1719:46tatutI mostly just want the :div.some-class to make classes, if that was provided builtin, that would be enough#2023-02-1719:46tatutfeel free to take that 😄#2023-02-1719:46Dustin Getz🙂#2023-02-1719:47tatutI often use tailwindcss so I want a convenient syntax for adding the static classnames#2023-02-1719:48Dustin Getzyeah the current dom syntax has been a frequently reported issue#2023-02-1719:48Dustin Getzwe agree with you btw#2023-02-1719:48Dustin Getzjust tricky to do right#2023-02-1719:49tatutthe compiler is giving me warnings for each of the content count cases like
90 | 0 (dom/element e (when attrs (dom/props attrs)) (doseq [[h b] handlers] (dom/on h b)))
------------------^-------------------------------------------------------------
hyperfiddle.electric-dom2/node not declared ^:dynamic
--------------------------------------------------------------------------------
#2023-02-1719:50Dustin Getzturn <% into defmacro not defn#2023-02-1719:50tatutI tried that first, but that didn’t work#2023-02-1719:50Dustin Getze/def is not visible in ordinary clojure, only from e/defn and e/def etc#2023-02-1719:50tatutif failed to analyze#2023-02-1719:51tatutbut I can try again#2023-02-1719:51Dustin Getzi will take a closer look over the weekend#2023-02-1719:54tatut(defmacro <% [elt & attrs-and-content]
(let [[attrs content] (if (map? (first attrs-and-content))
[(first attrs-and-content) (rest attrs-and-content)]
[nil attrs-and-content])
handlers (keep (fn [[key val]]
(when (str/starts-with? (str key) ":on-")
[(subs (str key) 4) val]))
(seq attrs))
classes (element-class-names elt)
attrs (cond-> (apply dissoc attrs (map first handlers))
(seq classes)
(update :class (fn [class]
(let [class-names (str/join " " classes)]
(if class
(str class-names " " ~class)
class-names)))))
e (element-name elt)]
`(dom/element
~e
~(when attrs
`(dom/props ~attrs))
~@(for [[h b] handlers]
`(dom/on ~h ~b))
~@(for [c content]
(if (string? c)
`(dom/text ~c)
c)))))
here’s the macro version, I must have made some mistake 1st time… now it worked#2023-02-1719:55Dustin Getzthere are also small syntax gaps that photon compiler doesn't understand yet, not all interop forms are handled correctly for example#2023-02-1719:55Dustin Getzi will play with it! thank you for sharing#2023-02-1720:34xificurCA minimal solution for simpler classes#2023-02-1804:04tatutthere seems to be a contrib namespace, I can make a properly documented version of that macro as a PR if you think it will be helpful for others#2023-02-1723:40JAtkinswhat’s the reason for extensive use of binding forms? Prolly a doc somewhere I missed I guess#2023-02-1723:44Dustin Getzi don’t think we’ve explained that yet#2023-02-1723:45JAtkinsmain e.g. I'm looking at:
(e/defn Main []
(binding [router/encode contrib.ednish/encode-uri
router/decode #(or (contrib.ednish/decode-path % hf/read-edn-str)
[`user.demo-index/Demos]
#_[[`user.demo-index/Demos . . .]]
#_{`user.demo-index/Demos {0 . 1 . 2 .}})]
(router/router (html5/HTML5-History.)
(set-page-title! router/route)
(binding [dom/node js/document.body]
(dom/pre (dom/text (contrib.str/pprint-str router/route)))
(let [[page & args] router/route]
(e/server (new (Pages. page #_args))))))))
#2023-02-1723:45Dustin Getzreactive dynamic scope is really cool, it is sufficient for dependency injection of the kind that DI libs like Component etc do#2023-02-1723:46Dustin Getzwe view it as composable DI basically#2023-02-1805:18tatutso instead of libraries like component or mount, you would bind a db connection pool at your app root?#2023-02-1808:52Geoffrey GaillardYes, these DI libs:
1. track dependencies to start/stop elements in the right order
2. Inject references in dependent components
Electric clojure does 1 out of the box.#2023-02-1805:01tatutwhat does it mean to get “server process crash” with
ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure #error {
:cause Cannot invoke "java.lang.Number.doubleValue()" because "x" is null
:via
[{:type java.lang.NullPointerException
:message Cannot invoke "java.lang.Number.doubleValue()" because "x" is null
:at [clojure.lang.RT longCast RT.java 1282]}]
:trace
[[clojure.lang.RT longCast RT.java 1282]
[hyperfiddle.electric.impl.runtime$parse_event invokeStatic runtime.cljc 756]
[hyperfiddle.electric.impl.runtime$parse_event invoke runtime.cljc 755]
[clojure.lang.PersistentVector reduce PersistentVector.java 343] ...
#2023-02-1805:03tatutOk, this particular case happened when I had an (let [foo (atom ..)] …) on the client side, and I tried to reset! that inside a e/server block#2023-02-1805:03tatutif I reformatted the reset as (reset! foo (e/server …)) it works#2023-02-1808:27Geoffrey GaillardThis is on us, sorry. Looks like a bug on unserializable references transfer (an atom can not be sent over the wire).#2023-02-1806:52tatutso how would one include the electric-jetty-server in an application, you would still need other routes apart from electric (authentication flows, file upload handling, even some APIs for other needs)#2023-02-1806:53tatutI guess you can build your own, ring handler#2023-02-1806:54tatutbut it would be convenient if start-server! took some config parameter that you could wrap the handler it generates with#2023-02-1808:46Geoffrey GaillardThank you for your feedback. We will improve the electric-starter-app server.
Electric just requires a jetty websocket endpoint to connect to. It can be plugged in any jetty setup. Ring is optional but convenient. You can take an existing ring jetty setup and add the websocket endpoint. Either:
• with the :websockets configuration map (jetty 9, https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_jetty_server.clj#L88-L89)
• with a ring middleware (jetty 10+, https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_jetty_server.clj#L43-L48)#2023-03-0222:37VinãoHi folks. I saw this message some time ago and did an experiment around this subject - I was particularly interested in this topic as well, but without time to tackle it timely.
Here is what I got:
• Using jetty9: https://github.com/viniciusalmeida/my-electric-start-app/blob/jetty9/src/user.clj#L16-L28
• Using jetty11 - the wrap-electric middleware turns the config REALLY easy: https://github.com/viniciusalmeida/my-electric-start-app/blob/jetty-upgrated/src/user.clj#L36
> We will improve the electric-starter-app server … Ring is optional but convenient
Do you folks already have a plan traced for these improvements @U2DART3HA? I understand the trade-offs of having or not ring as a dependency to run Electric, but it brings a really flexible way to set up web servers and it is widely adopted (IMO). If we can get/pick only this little middleware from the framework would be enough to have everything in place (on the server config, at least) and should allow developers to get started smoothly#2023-03-0309:51Geoffrey GaillardYou are right. The tradeoff is jetty 11 doesn't support java 8. We are still figuring out if java 8 compat is a requirement.#2023-03-0312:37tatutI guess java8 still has a long support and is running in many environments, but I would think that people aren’t starting new projects with it any more#2023-03-0313:01xificurCone can include electric in existing projects and move parts of the application incrementally#2023-03-0321:27Vinão> We are still figuring out if java 8 compat is a requirement.
You are not the only ones 🙂: https://clojurians.slack.com/archives/C8NUSGWG6/p1677871427282349 👀#2023-02-1810:43pezI got some help with wrapping my head around this language from @dustingetz, yesterday. Many thanks! Can't say I'm there yet, I probably must get my hands dirty quite a bit first. Anyway, calibrating my understanding here so that you can correct things I've gotten wrong. -> 🧵#2023-02-1810:44pezThe key things I need to get is this with Electric Clojure being reactive and this DAG. Of course, they seem to be very interconnected, it is the DAG that makes EC reactive. When I look at a piece of EC code I am actually looking at the DAG, the EC compiler builds the DAG from that code. Then it knows where in the graph each piece of (reactive) data flows. At run time, if some source of data has new content, each place in the DAG where this data is used can be updated.#2023-02-1810:47pezThe reaction is the result of a push, rather than a pull, and it is pushed only via the paths through the graph where there are things that need to react. If I squint I can see a the signal as it travels through the DAG. It looks like a bolt of lightning. Is this why it is called Electric Clojure?#2023-02-1810:51pezThat everything is reactive also means that it all is async. A call to a function will always be waiting. Is this true? An is it true even when the call is made to some piece of code that does not deal with any reactive data? Say (* (+ 1 2) 3). Will the * call be ”waiting” for the + to be ready? Or is it normal Clojure semantics there, that the inner function will be evaluated and then the outer, and there is no waiting going on?#2023-02-1811:02pezEvaluating a piece of electric code does not run it, right? It compiles it. So, in the editor, if I would evaluate som form inside an (e/defn ...), it only compiles it. I'll need to also run it, so I would use (e/run <some form>). This creates a program that will be running forever, waiting for signals through it's DAG, rerunning the program if they happen. Though this program is often small and cheap, it doesn't matter much if it keeps running. But in the REPL, we will often not expect to send anymore signals to it, so we should probably dispose of it. Fortunatelly, e/run returns a function that will dispose of the program. So we can (def dispose (e/run <some form>)) (dispose).#2023-02-1811:07pezIt is a bit unclear to me if this is guaranteed to actually wait for everything in <some form> to happen before (dispose) is evaluated?#2023-02-1811:11pezThere is currently no way to get the result of <some form> "out” to the Clojure REPL, so that it can be displayed by the REPL client. We will need to print it. So that would make have us evaluate (def dispose (e/run (println <some form>))) (dispose), something we can make our REPL clients do for us. (Calva has something we call REPL snippets, which could be used for this, and or course Calva could also build some of these in, once we've figured out what they should look like.)#2023-02-1812:17pezThis will only work if we know that dispose is not being called before we get to print the results. However, I think with RCF it is clear, since it is async. I tried with (tests (with (e/run (def r <some form>))) r), and it actually evaluates to the result of <some form> in the Clojure REPL. In the ClojureScript REPL it evaluates to nil, so there I need to do (tests (with (e/run (rcf/tap <some form>)))), and look for the results where stdout is printed.#2023-02-1812:44pezSince def has landed on master I can actually do things like this. 🤯 🎉#2023-02-1813:13Dustin GetzTranscoded to mp4#2023-02-1813:22Dustin Getz@U0ETXRFEW thank you for thinking out loud, this is deeply helpful to me as i learn how to explain this#2023-02-1814:08xificurCGreat writeup! Some quick notes:
• The model is actually pull, implemented in missionary. Child notifies of a new value and parent will transfer when it wants to. This isn't important for using electric, but it's nice to know as it solves problems of push models and allows implementing backpressure and laziness. Leo would have a better technical answer.
• Reactive values like e/watch compile to missionary and their composition creates a non-blocking reactive flow.
• * or + are clojure functions and they will run synchronously. If you have a blocking operation you'll wrap it with e/wrap to offload it on a threadpool.
• Currently an e/defn doesn't do much more than save the code as a sexp. e/run will macroexpand everything and do full program analysis and compilation.
• There is no waiting for a signal as it is defined in continuous time. E.g. how long would you wait for an e/watch ?
• rcf allows tapping a value onto a queue and % pops from the queue (waits for a value until a given timeout). This allows peeking into the program#2023-02-1814:28Dustin Getz@U0ETXRFEW to align this to our video chat:
• the push/pull dichotemy is stream vs signal
• Both stream and signal are reactive propagation strategies
• both stream and signal use the DAG to propagate updates
• the distinction is not that important. I'll write a blog about this
• Electric Clojure is about signals not streams#2023-02-1814:41teodorluabsolutely amazing explanation, @U0ETXRFEW - I needed to read this! Explains a lot of holes in my understanding; where I got stuck.
💯#2023-02-1817:42pezI now recorded it with mp4 instead of as a GIF. https://twitter.com/pappapez/status/1627000374887432196#2023-02-1817:43pezThanks for the feedback all of you! ❤️#2023-02-1819:38pezHow does the child know when to ask for a an update?#2023-02-1820:36xificurCCan you give an example of what you mean?#2023-02-1821:44vincentit's a web socket so if the value on the server changes it gets propagated to the connected clients, i believe#2023-02-1821:59xificurCif you meant how flows propagate here's a https://github.com/hyperfiddle/electric/blob/master/src-docs/user/missionary/missionary_flow.cljc#2023-02-1822:38pezI was referring to this:
> The model is actually pull, implemented in missionary. Child notifies of a new value and parent will transfer when it wants to.
Now checking the demo. 😃#2023-02-1911:05xificurCWe can take a call together tomorrow and clear this and the calva setup. These topics are hard to cover in chat#2023-02-1911:09pezThey are! Different timezones and stuff sometimes makes async communication, even if clunky, the best option. I'm happy to jump into a call. Let's try when you have the time.#2023-02-1911:31xificurCWe are in the same timezone, so I think we will find a time slot tomorrow. Will DM you#2023-02-1911:32xificurCI'll also try to write another explainer to further clarify#2023-02-1815:33Carsten BehringI started to use Electric and I am struggling to read a File in Clojurescript side (from a "file" <input>) and pass it (= the bytes) to my clojure functions on "server"
I am having the "File" object on clientt side, but have no clue how to go further.
I am lost between "atom" , promises, ".then" . Any help or example code appreciated.#2023-02-1815:44Carsten BehringMore concretely: I have
this code
(ui/button (e/fn []
(e/server (my-server-method a-byte-array)
)
#2023-02-1815:48Carsten BehringAnd I have an atom which contains a JavaScript promise of the ArrayBuffer for representing the file content.
How do I get a-byte-array to contain the file content ?#2023-02-1815:55vincentcheck out the starter app, i think you want a let after e/server and then a nested e/client#2023-02-1815:56vincenti am new as well so not 💯 sure#2023-02-1815:59xificurCAre you trying for the server to read a file and send its contents to the browser?
If you could provide a more complete example, or even a repo, that would speed up the process ;)#2023-02-1816:08Carsten BehringNo, the opposite. Having the client to read a file from user and send it to server.#2023-02-1816:11Carsten BehringSo this is wroking:
(dom/input
(dom/on "change"
(e/fn [e]
(let [first-file (-> e .-target .-files (.item 0))
my-array-promise (.arrayBuffer first-file)]
(swap! !state assoc :array my-array-promise))
nil))
(dom/props {:type "file"}))
So my "!state" gets the promise of an ArrayBuffer
And now I want that on "button press" I call a server function via
(e/server (do-somethng-with-bytes-of-file nytes-of-file))
#2023-02-1816:13Carsten Behring(ui/button (e/fn []
(let [bytes-of-file ......] ; get this from atom !state (-> !state :array .....)
(e/server (do-somethng-with-bytes-of-file bytes-of-file)))
nil)#2023-02-1816:23Dustin GetzLol, I dont know if Electric's network layer is the right fit for this, I would do file uploads out of band#2023-02-1816:24Dustin GetzI'm surprised you got this far#2023-02-1816:24Dustin GetzMaybe it can work, to be clear, why do you want to do this with Electric?#2023-02-1816:26Dustin Getzcan you paste me your entire working namespace as a gist, whatever you have so far#2023-02-1816:28Dustin GetzAlso, if you are using cursive, fix your indentation with this checkbox: https://github.com/hyperfiddle/electric/blob/master/docs/ide_cursive.md#2023-02-1816:39Dustin Getz& are you using a file uploader tutorial, if so can you link me to it#2023-02-1820:12Carsten BehringI am completely new to Clojurescript...and to Electric, so please forgive my ignorance...
But is surprises me that you ask me "why I use Electric ?"
Is Electric not a "full stack" Clojurescript framework, so it should support "file upload forms".
My "app" is a single form:#2023-02-1820:14Carsten BehringAnd I simply want that on "submit" the full form data gets "passed" to a server side Clojure function, including the fileupload data.
(which is a byte array or similar, I suppose)#2023-02-1820:14Dustin GetzMy mistake I misunderstood the request!#2023-02-1820:14Dustin GetzWe don't have a file uploader yet, sorry#2023-02-1820:14Dustin GetzIt's possible to integrate one#2023-02-1820:15Carsten Behringok. I see, so "integrate" with some existing reagent stuff ?#2023-02-1820:17Dustin GetzYeah and also would need to integrate the HTTP endpoint to accept the POST Content-Type: multipart/form-data#2023-02-1820:18Carsten BehringBut I am nearly there...
This does work, so I get access to the individual bytes.
So I could make an JavaScript array, so just miss to "transform" this to the Clojure / server side.
With "string" this works out of the box, but not with "arrays", it seems.
(ui/button (e/fn []
(-> (state :array)
(.then #(let [data-view (js/DataView. %)
b-0 (.getInt8 data-view 0)
b-1 (.getInt8 data-view 1)]
(prn :then-b0 b-0)
(prn :then-b1 b-1))))
nil)
(e/server
(azure/send-message ....))
(dom/text "Submit"))#2023-02-1820:20Dustin GetzWhat you are trying to do is very advanced#2023-02-1820:20Dustin GetzI would need to sync with the team to find the best approach#2023-02-1820:21Dustin Getzi have an idea though 1 sec#2023-02-1820:22Dustin GetzWhat is your level of clojure knowledge?#2023-02-1820:30Dustin Getzhttps://gist.github.com/dustingetz/8823e47c13f780d18938363d1d641b5b shows how to await a JS promise and bridge it to Electric Clojure#2023-02-1820:31Dustin Getzhowever if you get this working, it will fail on transit serializers for the byte array, we need to install a transit handler for it#2023-02-1820:34Dustin Getza cheap hack would be to base64 encode it and move to server as a string#2023-02-1820:34Dustin GetzI forget if we handle large payloads correctly today#2023-02-1823:35vincentmultipart params is a nice item for working on 😄 i think also uh what's it called, realtime seek on files, so you can scrub .mp4s ... arbitrary offset access for files... cannot think of the name right now x.x. but it would also be a nice one for future work#2023-02-1823:57Carsten BehringI got it working using your tips (await-promise and base64 encoding) as well for larger files.
👍#2023-02-1900:05Dustin Getzwow can i see it i can’t believe it worked#2023-02-1920:39Carsten BehringThe code of the button, which does the work is this:
(ui/button (e/fn []
(let [array (new (e/task->cp (await-promise (-> state :file (.arrayBuffer)))))
base64 (goog.crypt.base64.encodeByteArray (js/Uint8Array. array))
form-data (-> @!state
(dissoc :file)
(assoc :name (.-name (:file state)))
(assoc :file-content base64))]
(e/server
(e/wrap
(azure/send-message form-data))))
nil)
(dom/text "Submit"))
and the file <input>:
(dom/input
(dom/on "change"
(e/fn [e]
(let [first-file (-> e .-target .-files (.item 0))]
(swap! !state assoc :file first-file))
nil))
(dom/props {:type "file"}))#2023-02-1921:05Carsten BehringIs is somewhere documented which data types can be serialized between client and sever ?#2023-02-1921:36Dustin GetzIt's a default transit implementation, we need to make this dependency injected so you can control it, sorry#2023-02-1921:36Dustin Getzhttps://github.com/cognitect/transit-format#2023-02-1921:37Dustin Getzhere's a better table of default types https://github.com/cognitect/transit-cljs#2023-02-1921:43Dustin GetzLMK if you want me to rush in a fix#2023-02-2008:27Carsten BehringI have far too little experience in Clojurescript / reagent / Electric Clojure to suggest anything.
It seems to be mainly a lack of documentation question.
The "Network-transparent" feature of Electric is very magic and very cool,
but it should be documented which type of data it supports to "automatically move" between Clojurescript and Clojure#2023-02-2011:20Dustin Getzyou’re right we will document it thanks#2023-04-2520:20Rob B@U7CAHM72M I guess I'm not the only one trying to use Clojure/ClojureScript/Electric to make a GPT3/4 API App 🙂 this was a great thread to read.
@dustingetz I have never done "frontend" (let alone any Production Clojure) but I love the idea of using DAGs!e Electric makes sense to me.#2023-04-2520:56Dustin GetzDoes AI app need file upload?#2023-04-2521:47Rob BIf you want to solve the problem: “how do I build something like ChatGPT that uses my own data as the basis for its responses?” then yes.
LLMs (like GPT 3/4) are good at answering questions but if you want them to answer (reason) on your internal company data then you need to provide additional context to the model.
I'm trying to use Electric to build an end-to-end app that will:
1/ Allow the user to upload their data (pdf, text, csv) A button and upload a single doc is MVP, the enterprise approach is to have a batch/streaming data source integration.
2/ Parse the file into chunks (I'm currently trying to get https://github.com/techascent/tech.ml.dataset on the server side to work in Electric)
3/ With these chunks generate text embeddings (using Word2Vec for example)
4/ Save embeddings to https://www.trychroma.com/ or Postgres with https://github.com/pgvector/pgvector plugin (i.e. Vector store), maybe even Datomic if it has similar supporting features
5/ Then use something like your "ChatExtended" so that the user can ask questions to the LLM model in a chat window
6/ Then the AI app will retrieve the most relevant (convert user question to text embeddings and do cosine measure) prompts to the user input from the vector store using a similarity metric and generate the final (here is where the question and context are) prompt to send to the ChatGPT API and get a response in the AI app chat#2023-04-2612:26Carsten BehringI made a function for 2), which splits a file "optimal" for GPT so using the max (but not more) number of tokens in each split:
It is here:
https://github.com/BrewLLM/bosquet/blob/main/src/bosquet/splitter.clj
The moment I wrote it , I was not aware of any GPT-3 tokenizer working on the JVM,
so the tokenizer is pluggable and you need to provide your own tokenizer-f (which can then use python in some way to count the tokens of a given string)
Now there is one for the JVM, and there is some commented code how to use it together with the splitter function:
https://github.com/BrewLLM/bosquet/blob/0936df5ba312e199b49b8efc4bdba3d79a1f0d90/src/bosquet/splitter.clj#L45#2023-04-2621:38Rob BBosquet looks cool. I can learn a lot from this project (clj and patterns). Thanks for making it open-source!#2023-02-1820:47pezThis is probably not easy to follow in the recording... What is going on is that I have defined a Calva REPL snippet that looks like so:
{:name "Electric Eval Current Form"
:key "e"
:snippet (do
(tests
(with
(e/run
(def electric-repl-result $current-form))))
electric-repl-result)
:evaluationSendCodeToOutputWindow true}
It will wrap the current form in RCF and the resulting form will be evaluated. In the recording I am evaluating the forms (e/client (new (clock))) and (e/server (new (clock))) this way, first in the ClojureScript REPL, and then in the Clojure REPL. Good news is that it mostly works. But I don't quite understand a few things.
• When I evaluate e/client form (wrapped in RCF) in the ClojureScript REPL, I get the client clock as expected and it updates as expected. (Well, it still blows my mind, but anyway.)
• When I evaluate the e/server form in the ClojureScript REPL I get the client clock (as expected, because reader conditional), but it stops updating.
• When I evaluate the e/server in the Clojure REPL, I get some non-updating clock (not sure which clock it is)
• When I evaluate the e/client in the Clojure REPL, I get the server clock, and it is updating.
I'm pretty sure it is just me not understanding things. Can someone try to explain to me? Like I'm 5, please. 😃#2023-02-1820:49Dustin Getzthe last time we saw something like this it was a low level deadlock#2023-02-1820:51Dustin GetzYeah, the e/run is booting up additional "reactors" (that's the missionary event loop); it's not a configuration we test in#2023-02-1820:51Dustin GetzMaybe the m/sleep implementation has a low level concurrency bug#2023-02-1820:53Dustin GetzI think the assumption is there is only one missionary reactor ever running (per machine). If they have no interdependencies, they could never interfere but the clock is a source of possible interference#2023-02-1820:55Dustin GetzIndeed, https://github.com/leonoel/missionary/blob/b66b3e72827539dd672d08aa6490fe8a279cc251/java/missionary/impl/Sleep.java#L14 inside#2023-02-1820:55Dustin GetzTry a different example!#2023-02-1823:51pezIn many other example I get errors (in client repl) or just nil (server) when I evaluate with this RCF wrapper. The ClojureScript error message:
; Execution error (TypeError) at (<cljs repl>:1).
; Cannot read properties of undefined (reading 'call')
This happens with any form I try it with in the chat example for instance.#2023-02-1908:56xificurCwith will stop the program, so it is normal for the clock to stop updating#2023-02-1909:00xificurCe/run doesn't catch exceptions so even a Pending will nuke the program. Try catching at least pending and canceled https://github.com/hyperfiddle/electric/blob/master/test/hyperfiddle/electric_test.cljc#L1313#2023-02-1909:37pezThanks! Caching those seems to give me an unstuck clock for the case where I evaluate (e/server (new (clock))) in the Clojure repl. I expected it to sometimes return :pending... Maybe I am conducting my experiment wrong? I'll start over. 😃#2023-02-1911:29pezI think the problem with the RCF-wrapped evaluations from the ClojureScript REPL might be something with that RCF gets required/enabled in some funny state. I have a snippet for doing this, which looks like so:
[{:name "Electric Enable RCF"
:snippet (do (require '[hyperfiddle.rcf :as rcf :refer [tests tap % with]])
(rcf/enable!))
And often when I run it the first time for a namespace I get:
Execution error (ReferenceError) at (<cljs repl>:1).
rcf is not defined
And then I rerun that and might get the same error again, but eventually it runs without errors (returning true). But then it still often doesn't work to actually evaluate the RCF-wrapped forms, giving me that Cannot read properties of undefined (reading 'call'). Which right now happens for me in the Two Clocks example too, so it is probably not about a particular example.
In the Clojure it seems to always work.#2023-02-1912:47pezIt does seem that it is this going wrong in the cljs repl:
(do
(require '[hyperfiddle.rcf :as rcf :refer [tests tap % with]])
(rcf/enable!))
It only sometimes works. If I evaluate them one at a time, it always works.#2023-02-1912:55xificurCTry fully qualifying the enable call#2023-02-1914:02pezThat gets rid of the error messages for the require+enable. But it still only sometimes works. #2023-02-1823:04pezShould I understand this example? https://github.com/hyperfiddle/electric/blob/master/src-docs/user/missionary/missionary_flow.cljc I don't. 😃 🧵#2023-02-1823:04pezMaybe I am thrown off track already in the beginning.
> ; a signal is a "continuous flow" in Missionary jargon
> ; signals (flows) are recipes, like Haskell IO actions.
I'm not familiar with Haskell. But also, I don't know how to unpack the sentence (or if it is sentence_s_). Should it be read:
> A signal is a ”continuous flow” in Missionary jargon.
> Signals (flows) are recipes, like Haskell IO actions.
or
> A signal is a ”continuous flow”. In Missionary jargon
> signals (flows) are recipes, like Haskell IO actions.
It's OK to tell me that this is not important for starting to understand Electric Clojure. I can return to this later. But if it is important, then I'd certainly appreciate some help figuring the example out.#2023-02-1823:10pezCalva users could be interested in that RCF tests can be evaluated step by step using the command Calva: Evaluate from Start of List to Cursor, Folding Brackets, default bound to ctrl+alt+enter. It actually reruns all previous tests, but anyway.#2023-02-1823:33Dustin Getzno, this is very advanced#2023-02-1823:30vincentquestion, how can i repeat a dom element several times?
can I use a standard (`for [p [0 1 2 3]]` ... p) ?#2023-02-1823:34Dustin Getzuse p/for and p/for-by#2023-02-1823:34Dustin Getze/for and e/for-by sorry#2023-02-1823:34Dustin Getzthere are examples in the repo#2023-02-1823:37vincentexciting. so pretty much everything gets wrapped with electric macro so can DAG-dance?#2023-02-1823:37vincentand ty i'll have to look through all the examples, a appreciate this bounty of treasures#2023-02-1823:42Dustin Getzmostly just view things, queries you’ll still write in clojure #2023-02-1823:43Dustin Getzyou can also (do (dom/h1) (dom/h2)) to sequence dom elements#2023-02-2015:41Dustin Getz@U04PBP0MYS3 I misunderstood this question:
> pretty much everything gets wrapped with electric macro
#2023-02-2015:42Dustin GetzNo, most things are just ordinary Clojure, e/for is special#2023-02-2015:53Dustin Getz• ordinary for (clojure.core/for) works in Electric Clojure, but it is sequential, not concurrent/differential
• see https://github.com/hyperfiddle/electric/blob/d6f915bc4a50e1ba787a6438b6ad27713390da2c/src-docs/user/demo_3_system_properties.cljc#L31
• (e/for ...) desugars to (e/for-by identity ...) – identity is the "react key" used to stabilize the DOM, like React, Reagent, Om etc
• so that in the demo when you type into the input and the list adds/removes elements in response, it reuses the existing elements rather than rebuilding the DOM every time.
• That's what we mean by concurrent/differential: e/for-by is doing diffing to stabilize, and the children of the for loop evaluate concurrently
That's what e/for and e/for-by are.
Why did we provide a new operator for this instead of just extend clojure.core/for?
• It's not possible to extend clojure.core/for to be concurrent/differential
• Because we target 100% backwards compatibility with Clojure, we are unable to change the meaning of Clojure's for
• Pre-existing macros which use clojure.core/for in the macroexpansion need to continue working in Electric Clojure
• So we add e/for and e/for-by#2023-02-2015:54Dustin GetzMost Electric operators are backwards compatible and you don't notice them. if is reactive-if, try is reactive-try, binding is reactive-binding, let is reactive-let. Electric is able to extend all these operators without breaking backwards compatibility with Clojure#2023-02-1914:12Jungwoo KimQ) What does prefix ! mean in some examples…?
Here’s one of the examples.
(defonce !state #?(:clj (atom (list)) :cljs nil))
https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_4_chat.cljc#2023-02-1914:13Jungwoo KimWhat I mean is it has specific meaning like idiomatic clojure styles? something like this? https://clojure.org/guides/weird_characters#_symbol_unsafe_operations#2023-02-1914:14xificurCA convention marking stateful variables#2023-02-1914:16Jungwoo KimThanks! I’m familiar with (def ^:dynamic *state* …) though.#2023-02-1914:20Dustin GetzIn Electric, a common pattern is needing to separate a reference from the reactive value derived from the reference: https://github.com/hyperfiddle/electric/blob/eaa9f61b5856ff7b1317e6e1a85bd0181dc2d513/src-docs/user/demo_3_system_properties.cljc#L21#2023-02-1914:21Dustin Getzso you need distinct names for the two aspects of the state. The !x is a stable value - the identity needed to alter the state, and x is the ever-changing value of the state itself#2023-02-1914:22Dustin GetzIf you try to combine them, it causes reactive code that depends only on !x but not x to recompute more often than necessary, sometimes far more often#2023-02-1914:24Dustin GetzWrt earmuff notation, Clojure does something really unfortunate:
(e/def *x*) results in Warning: *x* not declared dynamic and thus is not dynamically rebindable, but its name suggests otherwise. Please either indicate ^:dynamic *x* or change the name.#2023-02-1914:24Dustin Getzso we can't use the earmuff notation really for anything; it's a shame#2023-02-1914:28Dustin GetzYou might ask, why not just pass around !x and call (p/watch !x) locally? That causes problems, it introduces "FRP glitches" (observable inconsistent states), because it results in multiple subscriptions to the same reference and clojure updates the subscriptions one at a time, causing your program to see a series of glitched states that eventually converge#2023-02-1914:31Jungwoo KimThank you for your detailed explanation. interesting. I’m getting more understands of states.#2023-02-1914:40Dustin GetzAny of us are happy to do a video call if you would like, we can accelerate your learning curve, would you like to do one?#2023-02-1923:00Jungwoo KimCool! ✋ I’d like it!#2023-02-2015:34Dustin Getz@U02SKL96W1J great - I will DM you in the next few days about doing the video call, i see you are in Asia TZ#2023-02-1914:44JAtkinsMorning 👋
I'm missing some subtlety right now. I've got a simple, but it's failing. Any time I click a button, I get this error in the js console. I'm not seeing any obvious difference between this example and the demo-2-toggle example though, so I'm not exactly sure what to look at for incorrect code.#2023-02-1914:46JAtkins#2023-02-1914:46JAtkinsha, I've always wondered if slack supported syntax highlighting. TIL#2023-02-1914:47xificurCYou mixed ui4 with old ui code#2023-02-1914:47xificurC(ui/button (e/fn [] ..))#2023-02-1914:47Dustin Getz::ui/click-event is not a thing anymore#2023-02-1914:47JAtkinsah. I think I copied from a dom/button example I found#2023-02-1914:48Dustin Getzwhere, i will delete it#2023-02-1914:48JAtkinsthere are a few. The one I saw was demo_bubbles.cljc#2023-02-1914:48JAtkinsL82#2023-02-1914:49Dustin Getzoh, scratch folder is a lot of old stuff#2023-02-1914:49Dustin Getzwe'll need to do some surgery on the repo#2023-02-1914:49JAtkins🙃#2023-02-1914:49Dustin Getzsorry for the confusion#2023-02-1914:55xificurCUndefined continuous flow happens when something that is not an electric function is called as such. In this case ui/button called a map#2023-02-1914:56JAtkinsSo, new ~V! is the culprit sorta? (electric-ui4/button)#2023-02-1914:58JAtkinsWill (e/fn [] (e/server (new Object))) cause trouble? (trying now)#2023-02-1914:59Dustin GetzI'm lost; the right way to call ui4/button is https://github.com/hyperfiddle/electric/blob/eaa9f61b5856ff7b1317e6e1a85bd0181dc2d513/src-docs/user/demo_2_toggle.cljc#L19#2023-02-1915:05JAtkinsSorry, I just worked out the confusion. I thought to get that error it was looking at (new {::ui/on-click ...}) and complaining specifically that the map was not an electric fn. Not the case, it's just complaining that new is not called on an object. I was overthinking the problem#2023-02-1915:05Dustin Getzoh, the confusion may be what new means here#2023-02-1915:05Dustin GetzElectric uses new to call electric functions#2023-02-1915:05Dustin GetzWe also capitalize them (like react components)#2023-02-1915:07Dustin GetzThe syntax sugar works –
(e/defn Teeshirt-orders-view [x] ...)
(Teeshirt-orders-view. x)
(new Teeshirt-orders-view x)
#2023-02-1915:07Dustin Getzdoes that resolve the confusion?#2023-02-1915:07JAtkinsYep, I was just reading the error wrong#2023-02-1915:07JAtkinsthankfully 🙂#2023-02-1915:36xificurCI think you had it right, JAtkins, ui/button's first argument should be an electric function and internally it called new on your map#2023-02-1915:00JHi guys! Does`e/defn` support multi arity?#2023-02-1915:00Dustin Getznot yet sorry#2023-02-1915:00JNo problem ^^#2023-02-1915:10JAtkinsAnother question - whenever I click the button it sets off an infinite loop and calls change-op! repeatedly. But, if I change to no args for change-op! , the loop is avoided.
Here's the modification to make it work:
(defn change-op!
[#_curr-op]
(let [curr-op (::op @!state)]
(println "updating-op - curr-op" curr-op)
(swap! !state assoc ::op (-> ops (set) (disj curr-op) (seq) (rand-nth)))))
(e/defn View
[{::keys [op]}]
(e/client
(dom/p (dom/text (str "Op is " (name op))))
(ui/button (e/fn [] (e/server (e/discard (change-op!))))
(dom/text "Change op!"))))#2023-02-1915:12Dustin Getzwhich snippet is which?#2023-02-1915:13JAtkinsThe complete snippit has the failing code, the one with change-op! no args works as expected#2023-02-1915:16Dustin Getzok i understand the issue#2023-02-1915:17Dustin GetzCan you read curr-op from the swap's provided prev value, rather than sampling the reference?#2023-02-1915:18Dustin Getzswap! !state update ::op ...#2023-02-1915:24JAtkinsYeah, that works. I'm just confused why the loop happens in case I run across it again 🙂#2023-02-1915:24JAtkins*rather how can I avoid it#2023-02-1915:24Dustin Getzit's subtle#2023-02-1915:25Dustin Getzlet me think how best to explain#2023-02-1915:25JAtkinscool no rush. I'm heading out - got church rn#2023-02-1915:25JAtkinswill be back in the afternoon#2023-02-1915:40Dustin GetzFirst, this is a known issue and we know how to fix it
1. change-op! will be called when any of its parameters change – per Electric semantics
2. e/server will throw pending until the remote result is known – per Electric semantics
3. e/fn is actually a DAG value or "piece of DAG"; it has a mount/unmount lifecycle – per Electric semantics
4. ui/button mounts the piece of DAG on click and then monitors for pending exceptions in order to unmount the piece of DAG when the pending state resolves
The loop is caused by latency; the swap! on the server -> the curr-op parameter to update on the server -> change-op! 's parameter has updated, so change-op! will be called again per (#1). And due to latency, the button doesn't unmount the piece of DAG (#4) until after (#1) has re-entered.
You shouldn't have to understand this, i will try to escalate the fix this week.#2023-02-1917:51JAtkinsAh, this makes a lot of sense. Thanks!#2023-02-1917:54xificurC(let [!x (atom 1)]
(reset! !x (inc (e/watch !x))))
#2023-02-1917:54xificurCCan you see the cycle?#2023-02-1922:43JAtkinsActually I’m back to confused. Why is the body of e/fn invoked more than once? I get that e/fn is a dag value, but I’m not sure why it’s contents are evaled #2023-02-1922:59xificurCIs this a question about the snippet above? Because there are no e/fns there#2023-02-1923:01JAtkinsI agree that snippet has a loop. Makes perfect sense. The confusion is how (or why) the body of the on click handler is replaced and called again. If it’s a bug that makes sense#2023-02-1923:02JAtkinsThe e/fn handler being a dag value that’s replaced also makes sense, the confusion is “why is it called again”#2023-02-1923:02JAtkinsI could be missing something here. It’s k inda guaranteed I am actually :)#2023-02-1923:03xificurCYour (change-op! op) has the same cycle#2023-02-1923:04xificurCIt changes op so it reruns, in a loop#2023-02-1923:04JAtkinsThe change op in the dag replaces the string rep in the dom, and recreates the click handler, if I understand right #2023-02-1923:05JAtkinsI’m stuck on a phone. I’ll try and make a picture shortly#2023-02-1923:05xificurCNot exactly, the click handler just never finishes running the e/fn because it loops#2023-02-1923:05JAtkinsOhhhhhh#2023-02-1923:13xificurCAs Dustin said we might be able to fix this case, just be wary of cycles like this for now#2023-02-1918:36JAnother question ^^. I try to send a message to the back part when the tab is closed. I use:
(dom/on "beforeunload" (e/fn [_] (e/server (prn "Foo") nil)))
But looks like the listener is not activated. I tried with the click event and it works.#2023-02-1918:42xificurCIIUC it should be mounted on the body#2023-02-1918:44JI use it like this:
(e/defn Room [room-id]
(e/client
(dom/on "beforeunload" (e/fn [_]...))
(dom/div ...)))#2023-02-1918:45Dustin Getz(binding [dom/node js/window]
(dom/on "beforeunload" (e/fn [_] (e/server (prn "unload") nil))))
#2023-02-1918:45Dustin Getz(tested, works for me)#2023-02-1918:47JThanks guys!#2023-02-1918:49Dustin Getzcleaned up: (dom/on js/window "beforeunload" (e/fn [_] (e/server (prn "unload2") nil)))#2023-02-1918:49Dustin Getzthere's another arity#2023-02-1918:50Dustin Getzhttps://github.com/hyperfiddle/electric/blob/6b7cee0e917f9ccfe98aa51faca1a7dbbf80bb9f/src-docs/user/demo_4_chat_extended.cljc#L60-L62#2023-02-1918:51Dustin GetzThe presence demo is detecting the disconnection from the server side#2023-02-1918:52Dustin GetzLet me know if you want an explanation of m/observe; it hooks flow construction and destruction lifecycle#2023-02-1918:56JI saw it (m/observe) in the code of the html5 router but on client side. Yes, I would like a little explanation about m/observe.#2023-02-1919:09Dustin Getzm/observe is meant to manage a resource - it lets you subscribe to a foreign reference (say a Clojure atom) and then release the resource when the flow terminates#2023-02-1919:10Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src-docs/user/missionary/missionary_tutorial_watch.cljc#L16-L24#2023-02-1919:10Dustin Getz(This is at the Missionary level so it's very low level)#2023-02-1919:11Dustin Getzhere, the point is that m/observe lets you detect the flow's cancellation signal and perform a side effect in response to it#2023-02-1919:13Dustin Getzhere I modified the toggle demo, to demonstrate using m/observe to hook the flow lifecycle from Electric:#2023-02-1919:14Dustin Getzthe electric if will switch between the two objects, unmounting one and mounting the other#2023-02-1919:17Dustin Getzfinally, in Electric we use new to join a foreign missionary flow value#2023-02-1919:18Dustin Getzthe parallels between objects and flows are on purpose, there is a very deep connection here#2023-02-1919:19Dustin Getzyou should almost always think of e/defn as defining a reactive function, and it absolutely is a function#2023-02-1919:20Dustin Getzyou can also think of e/defn as defining an object, with a resource lifecycle and state#2023-02-1919:20Dustin Getzit is both#2023-02-1919:21Dustin Getzthis is why we capitalize electric function names and call them with new#2023-02-1919:21Dustin Getz(same as React.js components)#2023-02-1919:24xificurCThere are also https://github.com/hyperfiddle/electric/blob/1c6c3891cbf13123fef8d33e6555d300f0dac134/src/hyperfiddle/electric_ui4.cljc#L122-L123 I once used to cover up the interop#2023-02-1919:28xificurCNot public API though, the ui4 namespace is still in flux#2023-02-1920:04JThanks guys! Why m/reductions is called with the m/observe?#2023-02-1920:06Dustin Getzm/observe builds a discrete flow, and electric clojure signals are continuous flows. The most important difference is that continuous flows must always have a value, so we use (m/reductions {} nil) to add an initial value, making it compatible with Electric reactive values which can never be undefined#2023-02-1920:07Dustin Getz{} here is pronounced "discard"; it is a function that discards the first parameter and returns the second. (Try it at the REPL and convince yourself it is true)#2023-02-1920:08Dustin GetzWhen using discard as a reducing function over a flow, it throws away the accumulator and returns the most recent value#2023-02-1920:12Dustin GetzAlternative implementation that avoids (m/reductions {} nil), perhaps this one is better#2023-02-1918:38vincentgm everyone ☀️
which examples should i read if i want to build a realtime multiplayer commenting and posting app#2023-02-1918:56Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_4_chat_extended.cljc shows presence, you'll need that#2023-02-1918:58Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_5_todomvc.cljc has nontrivial database queries, loading states, and interesting ux#2023-02-1918:39vincentin case anyone has a strong suggestion, if not i'll rummage 😅#2023-02-1920:48noogaI'm trying to wrap my head around this: When the app gets compiled into the client and server parts it must be that the client part is instantiated many times across users' browsers. What happens to the server part then? I imagine there is only one DAG on the server but there can be multiple flows going through at the same time. 0-cost Multiplayer would suggest that these flows can merge neatly, but how can they be kept separate?#2023-02-1920:50Dustin GetzGood question; on the server each websocket session gets its own instance of the server app. But all the sessions share memory space since they are on the same server#2023-02-1922:15Dustin GetzDatomic browser demo released: https://github.com/hyperfiddle/electric-datomic-browser#2023-02-2008:43Carsten BehringI have an question of "the absolute beginner", which knows nothing about Electric Clojure.
But this is maybe a key piece of documentation to be added first. For the rest we can look at the existing example code. Some words of explanation about the relationship between !x x and why / how and where we use them either as
(swap! !x not)
(if (e/server x)
This is key, because doing this wrong fails in (for a beginner) unpredictable ways, and not in obvious "compile errors".
(ns user.demo-2-toggle
#?(:cljs (:require-macros user.demo-2-toggle))
(:require
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
; A stateful app with a server-side counter
; Electric functions are reactive and incrementally maintained,
; which means that when you make a small change to the state,
; the functions will recompute and you'll get a small adjustment to the DOM
(defonce !x #?(:clj (atom true) :cljs nil)) ; server state
(e/def x (e/server (e/watch !x))) ; reactive signal derived from reference
(e/defn App []
(e/client
(dom/h1 (dom/text "Toggle"))
(ui/button (e/fn [] (e/server (swap! !x not)))
(dom/text "toggle client/server"))
(dom/p
(dom/text "Number type is: "
(if (e/server x)
(e/client (pr-str (type 1))) ; javascript number type
(e/server (pr-str (type 1)))))))) ; java number type```
By understanding this basics (and to know "which data types" are allowed in x), us beginners can do quite some experiments, to start to understand Electric Clojure and see its potential.#2023-02-2008:48Carsten BehringConcretely asked:
When am I supposed to read/change the "server state" !x and when am I supposed to read/change the "reactive signal" x
and why do I need this 2 ?
And why do I never need to "deference" the atom in the usual way @!x in this example app ?#2023-02-2009:15xificurC• We mark atoms with a leading exclamation mark, so !x holds a clojure atom which is a mutable data type, i.e. you can write a new value into it (`reset!` or swap!)
• (e/watch !x) returns a reactive read-only view of the atom's value
• foo in (foo (e/watch !x)) will re-run whenever the value of !x changes
• foo in (foo @!x) will only run once#2023-02-2009:38Carsten BehringOk, thanks, just one more question:
Inside both expressions
(e/server .... and (e/client
x can be accessed, as "in global scope`
Do I use them in exactly the same way across Clojure and Clojurescript and are they guarantied to contain the same data ?
(and what happens with data of different types in this case , maybe even incompatible between Javascript and JVM or Clojurescript / Clojure)#2023-02-2009:45Carsten BehringBecause it "seems" that x / !x is a common variable across client/server, to which Clojurescript can "write into" and Clojure can "read from".
Is this a correct interpretation ?
(Ok, the demo code above does no show the access of access to`x` from Clojurescript, but I have tried it and it seems to work)#2023-02-2010:06Carsten BehringThis variant of the toggle demo shows this magic very nicely:
(e/defn App []
(e/client
(dom/h1 (dom/text "Toggle"))
(ui/button (e/fn [] (e/server (swap! !x not)))
(dom/text (str "toggle client/server : current = "
(case x
true "Client"
false "Server"
))))
(dom/p
(dom/text "Number type is: "
(if (e/server x)
(e/client (pr-str (type 1))) ; javascript number type
(e/server (pr-str (type 1)))))))) ; java number type#2023-02-2010:42xificurCOne of the value propositions of electric is you shouldn't have to care where is the value defined to read it. Writing will happen on one side only though, you cannot reset! a server atom on the client (that would also be dangerous).
> Do I use them in exactly the same way across Clojure and Clojurescript and are they guarantied to contain the same data ?
yes
> what happens with data of different types in this case , maybe even incompatible between Javascript and JVM
currently we use transit to send data over the wire, so if you send something it cannot serialize you'll see an "Unserializable reference transfer" error in your console#2023-02-2010:33Dag NorbergWow, cannot overstate how cool this project is
Have been playing around with it for a while, wondering if you have any suggestions for deploying?
Tried putting the starter example in a docker image but if I try to deploy on something like http://Fly.io or Railway it runs out of memory before the server can start#2023-02-2011:23Dustin Getzwe will work on an optimized dockerfile that runs the Electric compiler at build time. it’s the compiler that uses too much memory#2023-02-2012:34Dag NorbergAwesome
If you are still doing zoom sessions I am very interested#2023-02-2016:48JAtkinsOk, I finally got to the example I was slightly worried about. It appears that e/defn calls create unbound vars outside the context of an electric fn. How would I go about making a dynamic data path, where parts of the reactive structure can be swapped out? Would that necessitate dropping down to missionary land? Does dropping down to missionary then prevent using the e/server and e/client because the compiler no longer is informed about context switches?#2023-02-2016:50Dustin Getzthe e/defn macro, today, just creates an ordinary def and saves the quoted Electric code as metadata on the clojure var, the whole program is compiled all at once at the application entrypoint#2023-02-2016:51Dustin GetzIIUC, for "dynamic data path" you have if statements which switch between pieces of DAG (imagine train track switch), as well as e/fn Electric lambdas which are higher order DAG values#2023-02-2016:52Dustin GetzDoes this answer your question? I'll provide an example after I am certain I understand the question#2023-02-2016:54JAtkinsSorta. It makes sense that this would work instead
(e/defn PerfOp
[{::keys [count op]}]
(println "PerfOp" count op (pr-str (get ops op)))
(new (case op
:times Times
:inc Inc) count))
but that's somewhat limiting. The use case I envision is using pathom to declaratively define the data model, and for components to declaratively define what data they depend on. Then a library would do the wiring up of a function call tree that provides the data required by the UI.#2023-02-2016:54Dustin GetzWe do not today support dynamic linking of Electric Clojure compilation units, so the "hyper-dag" (spanning set of all possible DAGs) must be known at compile time#2023-02-2016:56JAtkinshm... is this on the roadmap or are there technical limitations on doing something like this?#2023-02-2016:56Dustin GetzThis summer maybe#2023-02-2016:56JAtkinsI'm probably going to fiddle with it, but is it possible to write a subset of the dag in missionary and hook it into the electric compiler?#2023-02-2016:57JAtkins(probably through e/watch!)#2023-02-2016:57Dustin GetzYes you can join dynamic missionary DAGs but missionary DAGs do not have managed network, so no client/server transfer#2023-02-2016:58Dustin Getz> The use case I envision is using pathom to declaratively define the data model, and for components to declaratively define what data they depend on. Then a library would do the wiring up of a function call tree that provides the data required by the UI.
iiuc i see no reason this can't be done at compile time through a macro#2023-02-2016:58JAtkinsunrelated - is it possible to orchestrate multiple clients together? e.g. server, plugin, and client?#2023-02-2016:58Dustin GetzN-sites is on the roadmap#2023-02-2016:59JAtkinssick#2023-02-2017:00JAtkins> iiuc i see no reason this can't be done at compile time through a macro
Technically yes, it's possible to run that at compile time. However a step further on top of this idea is user-defined UI. electric takes care of a massive amount of UI wiring, pathom can take care of data path planning, so suddenly with only a bit more work you can get user defined UI. e.g. they could change a number input into a slider. Or, for a detail viewer show not only the name of a person but all user details.#2023-02-2017:01JAtkins#2023-02-2017:02JAtkinsthe image could use an update, but this is sorta the idea#2023-02-2017:03Dustin GetzYeah, this is the direction the Hyperfiddle product is exploring#2023-02-2018:18Dustin GetzRereading your example - to be clear, we have not just e/defn but also e/fn (lambda)#2023-02-2018:20Dustin Getzso in addition to
(e/defn PerfOp
[{::keys [count op]}]
(println "PerfOp" count op (pr-str (get ops op)))
(new (case op
:times Times
:inc Inc) count))
#2023-02-2018:21Dustin Getzwe also have
(e/defn PerfOp
[{::keys [count op]}]
(println "PerfOp" count op (pr-str (get ops op)))
(let [Times (e/fn [x] ...)
Inc (e/fn [x] ...)
Choice (if (= 1 (rand-int 2))
(e/fn [x] (* 10 x))
(e/fn [x] (inc x)))])
(new Choice count))
#2023-02-2018:21Dustin Getzit's really lambda, you can do quite a lot with this, the thing we don't have today is (1) eval and (2) dynamic linking#2023-02-2018:29Dustin Getz• Here is a test similar to your example, with https://github.com/hyperfiddle/electric/blob/4a5e43a630f542663ab561f110b5706d99a42d77/test/hyperfiddle/electric_test.cljc#L120-L128
• here's another good one - https://github.com/hyperfiddle/electric/blob/4a5e43a630f542663ab561f110b5706d99a42d77/test/hyperfiddle/electric_test.cljc#L1704-L1711 (Electric signals form Circuits actually not DAGs but we say DAG anyway for marketing reasons)#2023-02-2022:30JAtkinse/defn doesn't seem to play well with wrapping macros - I guess this is expected?
(defmacro wrap-defn
[name args & body]
`(e/defn ~name ~args ~@body))
(wrap-defn Thing
[m]
(dom/div (dom/text "Hello")))
;; macroexpand-1
#_(hyperfiddle.electric/defn Thing [m] (dom/div (dom/text "Hello")))
(e/defn App
[]
(e/client
(Thing. {})))#2023-02-2022:32Dustin Getzwhat error do you see?#2023-02-2022:32Dustin Getzi cant think of a reason that this wont work#2023-02-2022:32JAtkins[:dev] Compiling ...
[:dev] Build failure:
------ ERROR -------------------------------------------------------------------
File: /Users/jarrett/code/oss/electric/src-dev/user.cljs:7:20
--------------------------------------------------------------------------------
4 | hyperfiddle.rcf
5 | user-main))
6 |
7 | (def electric-main (hyperfiddle.electric/boot (user-main/Main.)))
--------------------------^-----------------------------------------------------
Encountered error when macroexpanding hyperfiddle.electric/boot.
Unable to resolve symbol: Thing
{:file "user/pathom_fulcroish.cljc", :line 85, :column 4, :in [(Thing. {}) (do (Thing. {})) (:hyperfiddle.electric.impl.compiler/client (do (Thing. {})) {:line 84, :column 3, :hyperfiddle.electric.debug/type :transfer, :hyperfiddle.electric.debug/name :hyperfiddle.electric/client}) (e/client (Thing. {})) (let* [] (e/client (Thing. {}))) (clojure.core/let [] (e/client (Thing. {}))) (:hyperfiddle.electric.impl.compiler/closure (clojure.core/let [] (e/client (Thing. {})))) (clojure.core/binding [hyperfiddle.electric.impl.compiler/rec (:hyperfiddle.electric.impl.compiler/closure (clojure.core/let [] (e/client (Thing. {}))))] (new hyperfiddle.electric.impl.compiler/rec)) (do (hyperfiddle.electric/-check-fn-arity! (quote App) 0 hyperfiddle.electric.impl.compiler/%arity) (clojure.core/binding [hyperfiddle.electric.impl.compiler/rec (:hyperfiddle.electric.impl.compiler/closure (clojure.core/let [] (e/client (Thing. {}))))] (new hyperfiddle.electric.impl.compiler/rec))) (:hyperfiddle.electric.impl.compiler/closure (do (hyperfiddle.electric/-check-fn-arity! (quote App) 0 hyperfiddle.electric.impl.compiler/%arity) (clojure.core/binding [hyperfiddle.electric.impl.compiler/rec (:hyperfiddle.electric.impl.compiler/closure (clojure.core/let [] (e/client (Thing. {}))))] (new hyperfiddle.electric.impl.compiler/rec))) {:hyperfiddle.electric.debug/name App, :hyperfiddle.electric.debug/args [], :hyperfiddle.electric.debug/type :reactive-defn, :hyperfiddle.electric.debug/meta {:line 82}})]}
ExceptionInfo: Unable to resolve symbol: Thing
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:781)
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:767)
hyperfiddle.electric.impl.compiler/eval31638/analyze--31644/fn--31649 (compiler.clj:826)
hyperfiddle.electric.impl.compiler/eval31638/analyze--31644 (compiler.clj:822)
hyperfiddle.electric/boot (electric.cljc:58)
hyperfiddle.electric/boot (electric.cljc:53)
clojure.core/apply (core.clj:671)
clojure.core/apply (core.clj:662)
cljs.analyzer/macroexpand-1*/fn--3704 (analyzer.cljc:4025)
cljs.analyzer/macroexpand-1* (analyzer.cljc:4023)
cljs.analyzer/macroexpand-1* (analyzer.cljc:4010)
cljs.analyzer/macroexpand-1 (analyzer.cljc:4074)
cljs.analyzer/macroexpand-1 (analyzer.cljc:4070)
cljs.analyzer/analyze-seq (analyzer.cljc:4107)
cljs.analyzer/analyze-seq (analyzer.cljc:4087)
cljs.analyzer/analyze-form (analyzer.cljc:4296)
cljs.analyzer/analyze-form (analyzer.cljc:4293)
cljs.analyzer/analyze* (analyzer.cljc:4349)
cljs.analyzer/analyze* (analyzer.cljc:4341)
cljs.analyzer/analyze (analyzer.cljc:4369)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4362)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/fn--2861/fn--2864 (analyzer.cljc:2010)
cljs.analyzer/fn--2861 (analyzer.cljc:2008)
cljs.analyzer/fn--2861 (analyzer.cljc:1931)
clojure.lang.MultiFn.invoke (MultiFn.java:252)
cljs.analyzer/analyze-seq* (analyzer.cljc:4080)
cljs.analyzer/analyze-seq* (analyzer.cljc:4078)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4085)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4083)
cljs.analyzer/analyze-seq (analyzer.cljc:4109)
cljs.analyzer/analyze-seq (analyzer.cljc:4087)
cljs.analyzer/analyze-form (analyzer.cljc:4296)
cljs.analyzer/analyze-form (analyzer.cljc:4293)
cljs.analyzer/analyze* (analyzer.cljc:4349)
cljs.analyzer/analyze* (analyzer.cljc:4341)
shadow.build.compiler/analyze/fn--18440 (compiler.clj:264)
shadow.build.compiler/analyze (compiler.clj:252)
shadow.build.compiler/analyze (compiler.clj:211)
shadow.build.compiler/analyze (compiler.clj:213)
shadow.build.compiler/analyze (compiler.clj:211)
shadow.build.compiler/default-analyze-cljs (compiler.clj:411)
shadow.build.compiler/default-analyze-cljs (compiler.clj:400)
clojure.core/partial/fn--5908 (core.clj:2642)
shadow.build.compiler/do-analyze-cljs-string (compiler.clj:321)
shadow.build.compiler/do-analyze-cljs-string (compiler.clj:278)
shadow.build.compiler/analyze-cljs-string/fn--18534 (compiler.clj:514)
shadow.build.compiler/analyze-cljs-string (compiler.clj:513)
shadow.build.compiler/analyze-cljs-string (compiler.clj:511)
shadow.build.compiler/do-compile-cljs-resource/fn--18562 (compiler.clj:629)
shadow.build.compiler/do-compile-cljs-resource (compiler.clj:610)
shadow.build.compiler/do-compile-cljs-resource (compiler.clj:568)
shadow.build.compiler/maybe-compile-cljs/fn--18665 (compiler.clj:961)
shadow.build.compiler/maybe-compile-cljs (compiler.clj:960)
shadow.build.compiler/maybe-compile-cljs (compiler.clj:936)
shadow.build.compiler/par-compile-one (compiler.clj:1081)
shadow.build.compiler/par-compile-one (compiler.clj:1036)
shadow.build.compiler/par-compile-cljs-sources/fn--18705/iter--18727--18731/fn--18732/fn--18733/fn--18734 (compiler.clj:1154)
clojure.core/apply (core.clj:667)
clojure.core/with-bindings* (core.clj:1990)
clojure.core/with-bindings* (core.clj:1990)
clojure.core/apply (core.clj:671)
clojure.core/bound-fn*/fn--5818 (core.clj:2020)
java.util.concurrent.FutureTask.run (FutureTask.java:264)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
java.lang.Thread.run (Thread.java:829)
--------------------------------------------------------------------------------
8 | (defonce reactor nil)
9 |
10 | (defn ^:dev/after-load ^:export start! []
11 | (set! reactor (electric-main
--------------------------------------------------------------------------------#2023-02-2022:36Dustin Getzthinking#2023-02-2022:39xificurCTry new Thing please#2023-02-2022:40Dustin Getzloading nice#2023-02-2022:42Dustin Getztry
#?(:cljs (:require-macros [user.demo-1-hello-world :refer [wrap-defn]]))
#2023-02-2022:43JAtkins[compiler] Still upset. Uncommenting the macroexpand-1 form makes it work again though.
(defmacro wrap-defn
[name args & body]
`(e/defn ~name ~args ~@body))
(wrap-defn Thing
[m]
(dom/div (dom/text "Hello")))
#_
(macroexpand-1 '(wrap-defn Thing
[m]
(dom/div (dom/text "Hello"))))
#_
(hyperfiddle.electric/defn Thing [m] (dom/div (dom/text "Hello")))
(e/defn App
[]
(e/client
(new Thing {})))#2023-02-2022:43Dustin Getzwe have an unknown shadow cljs issue likely related to macros and hot code reloading that we dont understand yet#2023-02-2022:43Dustin Getzyou may be hitting it#2023-02-2022:44JAtkinsha! that seems to have done it. Yeah shadow and macros are the bane of existence. I've spent a day or two debugging them [in the past]... not fun#2023-02-2022:44Dustin Getz#2023-02-2022:44Dustin Getzthis is modified from demo-1-hello-world#2023-02-2022:45Dustin Getzit works for me#2023-02-2022:45JAtkinsyup, can confirm#2023-02-2022:46JAtkins(honestly that's just regular cljs stuff that I should've figured out, but oh well - thanks for the help anyway 🙂)#2023-02-2022:46Dustin Getzwe are happy to get this kind of question for the time being at least#2023-02-2022:47Dustin Getzbtw what are you trying to do#2023-02-2022:47JAtkinsI'm screwing around with a fulcro-ish decorator for e/defn. Not very far along yet b/c I got stuck here#2023-02-2022:48JAtkinsI might be thinking too inside-the-box here, but the API might be nice#2023-02-2022:49Dustin Getzi dont know fulcro, what should i google to see what you're going for#2023-02-2022:49JAtkinshttps://book.fulcrologic.com#2023-02-2022:49Dustin Getzthis thing?#2023-02-2022:50JAtkinsThe first example sorta covers how it works. components are actually entities in a normalized database#2023-02-2022:50JAtkinsyep#2023-02-2022:50JAtkinsthe "normalized" bit makes data storage easier to reason about in my experience than something like reframe#2023-02-2022:51Dustin Getzclient database you mean?#2023-02-2022:51JAtkinsyep#2023-02-2022:51JAtkinsif it looks good I'll post a GH gist#2023-02-2022:57Dustin Getz#2023-02-2022:57Dustin Getz#2023-02-2022:58Dustin Getzi dont understand the query bit, which i suppose is the important part you want#2023-02-2022:58Dustin Getzthis has all the other lifecycle bits though#2023-02-2023:01JAtkinsThe query bit is important re. Prior thread about declarative data path planning only. The above might cover my needs though #2023-02-2113:03Dustin GetzDo you want to do another call and compare notes on this? We have a thing called "HFQL" which is comparable, its a bit too rough to show in public#2023-02-2113:04Dustin GetzI want to understand pathom better and especially how you're using it#2023-02-2113:10JAtkinsSure, sounds good. I'll put together some pseudo code for it beforehand, cuz our internal code is big and a mess 🙂#2023-02-2115:59Dustin Getz@pez i've switched to calva, how do i configure the indentation of fulcro-style dom to not align vertically?#2023-02-2116:01Dustin Getz#2023-02-2116:06pezThere is a setting to tell Calva where to look for cljfmt config file. This should be an edn file. See https://calva.io/formatting/#configuration#2023-02-2116:07Dustin GetzIs there a cljfmt rule that people use to do this?#2023-02-2116:09pezFrom there it is about adding the indentation rules into that file. You seem to want [[:inner 0]] for electric/dom. Maybe something like
{:indents {hyperfiddle.electric/dom [[:inner 0]]}
:alias-map {"dom" "electric/dom"}}#2023-02-2116:12pezIf the config file is within the workspace Calva will hotreload it. So you can have a :experimentation (or whatever) section in it and experiment within the file. (`tab` formats the current enclosing form).#2023-02-2116:15Dustin Getzgot it working, thank you#2023-02-2116:16pezWas the alias-map necessary? I'm guessing it might be because Calva only formats the minimal amount of text, so cljfmt doesn't have an ns-form...#2023-02-2116:31Dustin GetzI used {#"dom/.*" [[:inner 0]]}#2023-02-2116:32Dustin Getzhyperfiddle.electric-dom2/div [[:inner 0]] didn't resolve the aliases#2023-02-2116:32Dustin Getzdom/div did resolve the aliases but i need the regex anyway#2023-02-2116:32pezFYI: cljfmt and Cursive are quite incompatible around some default formatting so it can get a bit of a whitespace war with the require forms and some other cases. I've never quite figured it out. (And no-one else has either, for all I know.)#2023-02-2116:33Dustin Getzyeah we see that too#2023-02-2117:14Dustin GetzMade by @sr in Sweden, https://github.com/formicagreen/electric-clojure-painter/blob/master/src/app/canvas.cljc#2023-02-2117:30tatutso what would be the equivalent of mount/unmount in react like libraries… m/observe looks like it is used like that, but I don’t really grok it yet#2023-02-2118:45Dustin Getzyou can use these, i'll consider them for inclusion in the core ns#2023-02-2118:45Dustin Getz(defmacro on-mount [& body] `(new (m/observe #(do (% nil) ~@body (fn [])))))
(defmacro on-unmount [& body] `(new (m/observe #(do (% nil) (fn [] ~@body)))))
; usage
; note the body is ordinary Clojure, not Electric, so no transfers, side effects only
(on-mount (swap! !selected# select-if-first dom/node))
(on-unmount (swap! !selected# ?pass-on-to-first dom/node))
#2023-02-2118:46Dustin Getzdid you see this? https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/e_object.cljc#2023-02-2118:46Dustin Getz(from a thread yesterday)#2023-02-2204:35tatutthanks, that makes sense#2023-02-2209:01Tim Brown@dustingetz Electric is awesome from what I've seen! Good job.
I have a problem with the ReagentInterop demo: from http://localhost:8080/(user.demo-reagent-interop!ReagentInterop)
I get the following 😞
I've done an npm install, along with an upgrade of node to the shiniest (v19.7.0) version... any ideas?#2023-02-2211:23Dustin Getzyou have to uncomment it in i think user-main.cljc, it’s disabled to make the npm install optional#2023-02-2211:24Dustin Getzwe’re working on making the demos easier to run and find#2023-02-2211:24Dustin Getzfor now just uncomment the line locally#2023-02-2216:15Tim Brownthanks#2023-02-2214:57tatutPending looks cool, so I can use that to catch a situation where results are not available yet#2023-02-2214:57tatutwhat if I want to keep the previous DOM and show something in addition when Pending, I probably need a client side atom for the results?#2023-02-2214:58tatutlike a content that is updating, you don’t want to replace the old content with a loading indicator#2023-02-2216:11JAtkinsmy initial thought would be maybe priority stream pulls from missionary land..? But I'm not sure how to do that yet, and there's probably a better method#2023-02-2216:15Dustin Getzexceptions leave the try-body in place and running (so the dom stays, the catch-body is meant to add a loading on top of what is already there)#2023-02-2216:15Dustin Getzthis is because when the exception "goes away" you need to resume where you were#2023-02-2305:01tatuthmm, maybe the structure of my code is just wrong#2023-02-2311:17Dustin Getzsend me a gist of what you have and a comment of what you want to happen #2023-02-2314:17tatutI’ll try, I’m not sure what I “want” to happen, as I’m more trying to understand how to use#2023-02-2314:32tatutthis is one thing I tried, having a catch block in the UI https://github.com/tatut/electric-examples/blob/55e7537544aefeed268354a97d5fa8d5fc9051e6/src/app/ankka.cljc#L99 this looks like it the “loading” block was always showing#2023-02-2314:34Dustin GetzAre you running master? I think we might have a regression that matches this report#2023-02-2314:34tatutI updated yesterday, I’ll try with latest#2023-02-2314:35Dustin Getzare you blocked on this?#2023-02-2314:35tatutno#2023-02-2314:35tatutjust doing some examples as I’m going to show some electric to other clojure programmers at work tomorrow#2023-02-2314:35Dustin Getzok#2023-02-2314:36Dustin Getz";; This is showing always after initial showing,"
i think you are doing it right, this is a bug
We refactored how exceptions work this week#2023-02-2314:38Dustin GetzI think if you revert to the published version (v2-alpha-68) it will work properly#2023-02-2314:40tatutI’ll try reverting#2023-02-2314:48tatuteven after reverting it seems that it doesn’t work properly, not it isn’t showing all the time… but the pending element only very briefly flashes in existence and is removed#2023-02-2314:49tatuteven when I added a 2 second thread sleep in my jdbc query function to really see it#2023-02-2314:50tatutif I put a println in the catch block, it is printed twice, once at the start and again at the end of the wait when the element flashes#2023-02-2314:54tatutah no sorry… the element flashes twice when the waiting starts and after the wait is over#2023-02-2314:59Dustin Getzok, we will look into it, thanks for the report#2023-02-2315:00Dustin Getzyour latest code is on github?#2023-02-2315:00tatutI’ll push this now#2023-02-2216:10JAtkinsI don't think this is important to the code, but what's the frequent use of :needle?#2023-02-2216:13Dustin Getzneedle in a haystack ... sorry will stop doing that ty for asking#2023-02-2216:34JAtkinsnp, just curious if it had any interesting meaning#2023-02-2216:46Adrian SmithThere's an english? Saying for something that is hard to find
"It's like finding a needle in a haystack", in programming it's commonly translated to needle is the thing that you are searching for and haystack is the thing being searched#2023-02-2216:57JAtkinsYup. Sorry yeah it made sense as soon as I saw "needle in a haystack" 🙂#2023-02-2217:33JHi guys! Does electric support release code with shadow-cljs?
The builds of shadow-cljs.edn config:
:builds {:app {:target :browser
:output-dir "resources/public/js"
:asset-path "/js"
:modules {:main {:init-fn }}
;; For dev
:dev {:devtools {:watch-dir "resources/public" ; live reload CSS
:hud #{:errors :progress}
:ignore-warnings true}}
;; For prod
:release {:compiler-options {:optimizations :simple}}}}
But when I pack the uberjar, the server starts but the js log this error in the console. I miss something?#2023-02-2218:22Dustin Getzwe do support advanced optimizations, let me get back to you#2023-02-2218:43nakkayaI was playing with the same idea, hit the same error. removing optimizations fixed the error. However now when I run the uberjar I get unable to resolve symbol error related to my backend functions. Am I skipping something?#2023-02-2218:50Dustin Getzmaybe we broke it, builds are getting reworked wip rn#2023-02-2220:12nakkaya@dustingetz Do you happen to know any repo that shows how to deploy an electric app with or without a jar? Just to cross check what I have.#2023-02-2309:37Geoffrey Gaillard:optimization :simple has been fixed.
:optimization :advanced has a regression. We will address it.
> when I run the uberjar I get unable to resolve symbol error related to my backend functions
>
We are addressing this too. We’ll get back to you with an example of how to deploy an electric app with a jar.#2023-02-2312:41JThanks @U2DART3HA it works and I have a valid uberjar too!#2023-02-2312:42nakkaya@UHZPYLPU1 Any chance you can share the repo? I am still struggling with building the uberjar.#2023-02-2312:53JI have not repo yet @U5H4U2HEH 😅 but I can describe how I manage the uberjar.
This is the view of my src folder:
src
├── client
│ └── estimaty
│ ├── app.cljs
│ ├── routes.cljs
│ └── util.cljs
├── logback.xml
├── main
│ └── estimaty
│ ├── core.cljc
│ └── views
│ ├── header.cljc
│ ├── room.cljc
│ ├── rooms.cljc
│ └── signup.cljc
├── server
│ └── estimaty
│ ├── db.clj
│ ├── repository.clj
│ └── server.clj
So src/client , src/server and src/main are source folder (`estimaty` is the main namespace).
In my deps.edn , the dev aliase looks like this:
:dev
{:extra-paths ["dev" "src/client" "src/server" "src/main"]
:extra-deps
{binaryage/devtools {:mvn/version "1.0.6"}
thheller/shadow-cljs {:mvn/version "2.20.1"}}
:jvm-opts
["-Xss2m" ;
"-Dclojure.tools.logging.factory=clojure.tools.logging.impl/slf4j-factory"
"-Dlogback.configurationFile=src/logback.xml"
"-XX:-OmitStackTraceInFastThrow" ;; RCF
"-XX:+UnlockDiagnosticVMOptions"
"-XX:+DebugNonSafepoints"]}
Now this is the build.clj to build the uberjar:
(ns build
(:require [clojure.tools.build.api :as b]))
(def lib 'io/estimaty)
(def version "0.0.0-alpha0")
(def class-dir "target/classes")
(def basis (b/create-basis {:project "deps.edn"}))
(def uber-file (format "target/%s-%s-standalone.jar" (name lib) version))
(defn clean [_]
(b/delete {:path "target"}))
(defn uber [_]
(clean nil)
(b/copy-dir {:src-dirs ["src/server" "src/main" "resources"]
:target-dir class-dir})
(b/compile-clj {:basis basis
:src-dirs ["src/server" "src/main"]
:class-dir class-dir})
(b/uber {:class-dir class-dir
:uber-file uber-file
:basis basis
:main 'estimaty.server}))
The server should know your server function.
Maybe there is a better solution ^^.#2023-02-2313:10nakkaya@UHZPYLPU1 mine is pretty much same as yours. I do get the uberjar and it loads fine, my problem occurs when I try to execute backend functions with e/server I get symbol can not be resolved error. Did you had to change anything in front end init code? In my case I split user.clj in to two, dev still launches user.clj but jar uses core.clj that has jetty start logic without shadow-clj stuff. Similarly I moved user.cljs to core.cljs and updated shadow edn accordingly.#2023-02-2313:11J> I get symbol can not be resolved error
This occur on the server, right?#2023-02-2313:11nakkayayes#2023-02-2313:12JHave you required your backend function to the entry point of your uberjar?#2023-02-2313:14nakkayaWhen you say backend function, I have a core/-main that jar executes which starts jetty, or do you mean something else?#2023-02-2313:17JIn my example, my -main function is into the server.clj . In this file, I have required the backend namespaces used by all the views (like db.clj, repository.clj)#2023-02-2313:33nakkaya@UHZPYLPU1 lol, looks like I fixed it without realizing I fixed it. I required db ns to handle authentication in my jetty ns. Like you pointed that fixed the issue but at that point last night I already gave up on the jar so never tested again. Thank you.#2023-02-2310:44Yab MasI’m trying to get my head around the consequences of a stateful service for (horizontal) scaling / deployment options in general.
We’re currently deploying most apps to google cloud run. Mainly because it’s a cost effective/easy way to deal with the occasional spike in traffic. We would preferably deploy electric-apps this way as well. Cloud run offers sticky connections, but it’s best-effort and data should be synchronised between containers to make it fault-proof.
It’s not clear to me which data I need to sync to make this work with an electric-app. If it’s just the references that are turned into signals with e/watch I believe our case is not that complicated. We could use something like “firestore real-time updates” to keep this in sync (firestore is our main db). But again my understanding is not complete and maybe I would need to sync more state to make this work. Can you elaborate?#2023-02-2311:19Dustin Getzi think if you start with the chat demo, which uses an atom for state, and then substitute the atom with a virtual atom backed by firestore, it will just work#2023-02-2311:20Dustin Getzso long as the database fans out updates to all connected clients#2023-02-2312:11Yab MasPerfect. That's what I had in mind. I won't be testing this anytime real soon, but I'll report back or aks questions once I get to it.#2023-02-2312:19Yab MasWell, not fully what I had in mind actually. You're saying I should fan out to all clients, I was thinking to update the (virtual) atoms of de servers/containers, which would than update their connected clients. Is that idea not correct? Anyway, the details don't matter that much atm. I'll try some things when I get to that point. Right now it's good enough to know the whole thing isn't doomed to begin with.#2023-02-2319:37Dustin Getzi dont understand the distinction you are making#2023-02-2320:37Yab MasWhen you say "so long as the database fans out updates to all connected clients" - by "clients" do you mean: firestore-clients which are electric-servers or electric-clients? If its the former we mean the same thing.#2023-02-2401:12Dustin Getzyes the former#2023-02-2314:19Carsten BehringAn other absolute beginner question:
How can I get from https://github.com/hyperfiddle/electric-starter-app
to a working uberjar ? (runnable via java -jar ....)
I added a build.clj to make a jar, but runnnig the jar with the code as is, fails.
I probably complete miss "release" build config / setting /code and have nver used shadow-cljs before.
I probably need to stop to "watch" and go to "compile", but I have no clue on this.
Is there simple documentation somewhere ?#2023-02-2314:21Jhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1677156822710569?thread_ts=1677087230.764779&cid=C7Q9GSHFV#2023-02-2314:22JFor the shadow-cljs part, you need to use the release command of shadow like shadow-cljs release <your-build>#2023-02-2314:28Dustin GetzWe're working on instructions for this, give us a few more days#2023-02-2317:18Dustin Getz> First try using "Electric https://twitter.com/hashtag/Clojure?src=hashtag_click"
> Multiplayer Sudoku in <150 LOC, ~6h. Pretty neat!
> https://github.com/s-ol/electric-sudoku/blob/main/src/app/sudoku.cljc
> https://twitter.com/S0lll0s/status/1628798014042083328#2023-02-2318:17Dustin GetzPacific TZ zoom onboarding today Feb 23, 3pm - 5pm PT – two slots open, DM me#2023-02-2318:34denikThese print on every shadow reload adding noise to repl output. any way to disable?#2023-02-2318:35Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/main/src/logback.xml#2023-02-2318:35Dustin Getzprobably it got deleted or moved off classpath#2023-02-2318:37denikah yes, still had photon config#2023-02-2318:37denikthanks!#2023-02-2318:37Dustin Getzah#2023-02-2320:01JHave you seen this https://twitter.com/manucorporat/status/1628758078572175360?s=46&t=CHkGEJYls38ixVHi0i-aqw#2023-02-2320:30Dustin Getzi saw it today, couldn’t find any details#2023-02-2401:32Dustin Getzhttps://twitter.com/RyanCarniato/status/1628891152052281344#2023-02-2412:05Jhttps://twitter.com/tannerlinsley/status/1629014767804502022?s=46&t=uJuIqVCfVMm9hSFQN8edLg#2023-02-2321:29nakkayaIs it possible to access session key from within e/server ? What I am trying to is to decide if the user is allowed to perform an operation. When the user is logged in, a cookie is set I would like to access the cookie to check user role.#2023-02-2321:36Dustin Getzlike this? https://github.com/hyperfiddle/electric/blob/9be03dd66f012971716544aa56c197bf5d0a85b2/src-docs/user/demo_4_chat_extended.cljc#L46#2023-02-2321:38nakkayaI tried that but even though hf/*http-request* contains the cookie it is not decoded.#2023-02-2321:38Dustin Getzi dont understand#2023-02-2321:42nakkayaWhen I do, (e/server (println hf/*http-request*)) the map contains the session key however it is not decoded,
cookie ring-session=2aAYU8Xq3YcxHNRGA1ElgYfi%2BZw7vhE5g7%2F1fJk%2FI7roVe1jmZzSapmWx1s30bRZd4cj%2F13Jky38zO8zeTAfK%2ByXobr87vjDZPd3CGEhcfhbUgzDZ%2BDIzZnl8FQ9Onjqjyxt%2FyJWUhUs0yVeGpMUGlCvCuah51bE2n%2FTw4EpXdY8xEpK634pVzxAnr6n5KxJrGWR8LfolZZVy3AfN%2BWjgUYbRTuLsPFuGldpB8CRrWmkYDSYgP35Ia%2BUtIWIdth3--EqsuKkVZPK3xmg%2FJQ0YA%2F%2BE9NHWumcT9qWKUa4jfUGo%3D
#2023-02-2321:43Dustin Getzcould a middleware be running in the wrong order? I dont think this is an Electric issue#2023-02-2321:44Dustin GetzI don't understand who is encoding the value, it isn't us#2023-02-2321:44Dustin Getzi could be confused#2023-02-2321:45nakkaya> could a middleware be running in the wrong order?
It could be on my side. I'll check. Is there a more idiomatic way to achive what I am trying to achieve? I still need the cookie for my other routes such as file upload but for electric side I can go with something else.#2023-02-2321:46Geoffrey GaillardThis looks like the ring-session middleware#2023-02-2321:46nakkayacorrect#2023-02-2321:48nakkayaI slightly modified jetty start code for my needs,
(cond-> #'default-handler ; these compose as functions, so are applied bottom up
true (wrap-file resources-path) ; 6. serve it
true (wrap-content-type) ; 5. detect content
true (wrap-default-page) ; 1. route
true (wrap-no-cache) ;; TODO disable in prod
true (wrap-authentication auth-backend)
true (wrap-authorization auth-backend)
true (wrap-defaults (assoc-in site-defaults [:session :store]
(cookie-store {:key "a 16-byte secret"}))))#2023-02-2323:07Geoffrey GaillardIIRC, ring-session reads the cookie in the request header and gives you a :session key in you request map.#2023-02-2323:25nakkayacorrect I have access to it in my ring handlers problem is :session key populated.#2023-02-2323:31Geoffrey GaillardYou are right. I think I got it.
We are using jetty 9 (for java 8 compat)
Jetty 9 separates declaration of http handlers from websocket handlers.
The ring session middleware should be applied to both http and websocket declarations.
It seems it is only applied to http in your case.
Have a look here https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_jetty_server.clj#L89-L94#2023-02-2323:46nakkayaThanks that was it, for anyone that stumbles this the solution was,
:websockets {"/" (fn [ring-req]
(adapter/electric-ws-adapter
(partial adapter/electric-ws-message-handler
(-> ring-req
(session/session-request
{:store cookie-storage
:cookie-name "ring-session"})))))}#2023-02-2322:34Carsten BehringI put my Electric app on "https" and know I get errors that it cannot connect to websocket on "ws" (not "wss"):
main.js:1426 DOMException: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
Is there a way to change the config and let it do secure websocket connections ?#2023-02-2322:46Geoffrey GaillardThe websocket should connect with wss if the page protocol is https by default ( https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_client.cljs#L8-L19)
I would like to try to reproduce the issue. What tool do you recommend me to get it to serve over https quickly?#2023-02-2323:11Carsten BehringI saw the code, which should do this automatically, but somehow it does now work in my case.
I use Azure Container Apps, which does among other thinks a http > https#2023-02-2323:15Carsten BehringI had this line before, thats probably the real l error:
websocket.cljs:12 Mixed Content: The page at '' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint ''. This request has been blocked; this endpoint must be available over WSS.
#2023-02-2323:33Geoffrey GaillardOk thank you for the details. We'll look into it#2023-02-2412:00Geoffrey GaillardFound it!
The electric client will correctly switch to wss if served from an https page. No bug here.
The failing connection we can see in your error message is shadow-cljs (cljs compiler) dev tools trying to connect to the server.
The reason is you are shipping a dev build to your Azure server.
You should use the (release)command (in user namespace, next to main) to build a production .js file and ship this.#2023-02-2516:51Carsten BehringThanks for the hint.
I was a bit in a rush for deployment, so just copied all to a Docker...
I was somhow aware, that this was not a proper "release build",
I will try that. I am still wondering "what exactly" I need to ship, to have fronted and backed working.#2023-02-2516:52Carsten BehringI read you work on some release scripts, so wait for them.#2023-02-2322:51refsetCan Electric help with running synchronous-looking client-side code on top of async browser APIs? e.g. cheaply solving hard problems like https://github.com/tonsky/datascript/issues/190 - have there been any examples created in that vein already?#2023-02-2323:03Dustin GetzI dont understand the question?#2023-02-2323:37refsetExpanding on the example, DataScript is written entirely assuming synchronous function calls (against in-memory data structures), so naively adapting it to use callback-based storage underneath (for browser ClojureScript) would be very invasive and affect a tonne of code & APIs. I am wondering whether Electric has some ability to help in this scenario...like compiling away local async I/O introduced by IndexedDB, such that the Electrified version of DataScript isn't bothered about the difference.#2023-02-2401:11Dustin GetzElectric Clojure can be used to build an async Datascript (and this is probably worth doing). A POC can probably be built quickly, but it will not be free#2023-02-2411:02refsetCool, thanks for confirming my understanding. Making local I/O simpler is only moderately compelling in isolation (full-stack is obviously the game-changer), but still I bet there are loads of purely frontend ClojureScript codebases out there that could (in the fullness of time) benefit from switching to Electric for this capability alone :thinking_face:#2023-02-2411:06refsetI should add that I don't actually have any specific need for async DataScript, it's just a concrete example of something that's previously been considered hard/impossible.#2023-02-2411:20Dustin Getzimagine reactive datascript with client/server transfer, the query engine and index data structures coded in electric and therefore can distribute over network such that last mile filtering can happen on the client. offline first!#2023-02-2411:32refsetAh yes, I saw you mention this distributed DataScript possibility already somewhere - that would certainly be next-level compelling 🙂#2023-02-2417:54denikalso electric-wrapped-async-datascript in a webworker#2023-02-2323:10vincent[:user.demo-index/chat-extended] has a reference to /auth ...#2023-02-2323:11vincentwhere is that x)#2023-02-2323:11vincentthe implementation of that i mean#2023-02-2323:12vincentnvm found it in electric_jetty_server#2023-02-2323:26vincentwishlist: reusable components library#2023-02-2410:10Geoffrey GaillardWe are having an Electric Clojure Zoom session onboarding.
We answer questions and help you solve issues.
Anyone feel free to attend. DM me.#2023-02-2414:41tatutwhy is Pending an exception? is it just due to convenience of using try/catch for that? having to wait for a call isn’t really exceptional or an error#2023-02-2414:50Dustin Getzthe semantics matched#2023-02-2414:51Dustin GetzIts probably best to think of Electric as its own language, the idioms are different than Clojure#2023-02-2414:53Dustin GetzAnother example - in Clojure dynamic scope has a bunch of problems - it interacts badly with clojure's lazy collections, and also from a JVM concurrency perspective, bindings are threadlocals ~which have terrible operational semantics~ [Leo says: The reason why dynamic vars have terrible operational semantics and bad performance is the binding conveyance mechanism; there is nothing wrong with threadlocal which has well defined operational semantics and are also fast]
but in Electric, dynamic scope doesn't have any of those problems, reactive bindings are a foundational compiler primitive and something we use extensively#2023-02-2414:57tatutthanks for the explanation#2023-02-2415:03xificurCanother way to think about it - in electric everything has to have an initial value. What's the initial value of a server block that didn't come yet?#2023-02-2415:09tatutcoming back to my Pending loading indicator problem from yesterday, why is e/wrap needed for blocking JVM calls? results seem to work without it#2023-02-2415:09tatutbut it does show the catch block better if I add wrap#2023-02-2415:10Dustin Getzblocking calls will block the event loop#2023-02-2415:10Dustin Getzin a multi-user scenario all events will be blocked#2023-02-2415:10Dustin Getzin our datascript demo we cheated and omit the e/wrap, we should fix it#2023-02-2415:11tatut“all events” as in all events for all connected clients?#2023-02-2415:13tatutbut now that makes sense why the loading indicator didn’t work when I wasn’t using wrap#2023-02-2415:13xificurCit completely blocks the server side until it finishes#2023-02-2415:14xificurCeach client has their own server event loop, so not all clients#2023-02-2415:15xificurCe/wrap throws Pending so yes, it should improve the demo#2023-02-2415:16Dustin Getz@U09FL65DK do they all have threads?#2023-02-2415:16Dustin Getz@U11SJ6Q0K btw pull master , we fixed a regression related to Pending#2023-02-2415:16Dustin Getzor i can publish a new version#2023-02-2415:16tatutI’ll sync my fork#2023-02-2415:17xificurCre threads, not sure#2023-02-2415:19tatutone thing I liked about Phoenix LiveView (well many BEAM based systems) is the cheap processes and every connection having its own isolated process loop#2023-02-2415:19tatutbut perhaps the JVM isn’t as good (at least not before project loom)#2023-02-2415:24xificurCYou're right, BEAM has an advantage here. Loom will make this advantage smaller, but the JVM is still built for throughput#2023-02-2415:24xificurCEither way electric needs to run async, so all blocking calls should be wrapped in e/wrap#2023-02-2415:27tatutI could imagine wrap calls to be needed very often, perhaps a e/server-wrap convenience (kind of like async has go-loop)#2023-02-2415:59Dustin Getzyeah, could be. Maybe the Electric compiler can do stuff automatically, idk#2023-02-2416:04tatutif you would always just wrap everything, would that cause problems? too much pendings even when you don’t need them#2023-02-2416:14Dustin Getzone issue with wrapping everything is that calls like inc and println are indistinguishable from query-database , they are all synchronous and blocking#2023-02-2416:14Dustin Getzso the cost of moving inc to a threadpool is paying thread communication overhead which dwarfs the cost of the actual inc#2023-02-2416:16Dustin Getzone idea is to use statistical heuristics, the "Electric VM" ~can~ could maybe detect slow calls#2023-02-2416:17tatutprobably not worth it atm#2023-02-2416:17tatutbut I was thinking about electric vs traditional apps#2023-02-2416:17tatutusually you have some logging and middleware in backends#2023-02-2416:18tatutcould you do something like that in electric, like bind a call fn that takes a server side function as parameter to call it, wraps it handles errors and logging… even catches exception and shows “error topbar” on client#2023-02-2416:19Dustin Getzcan you give me a concrete example of that in the context of one of our demos, say the todo list demo?#2023-02-2416:29Dustin GetzOk I understand now - you just want a easy way to log queries, intercept errors, and route errors to a banner#2023-02-2416:30Dustin Getzi think you could code that today in userland in a few LOC with a higher order electric fn, or a macro#2023-02-2504:24tatutexactly that, I’ll try to experiment with that#2023-02-2504:25tatutmiddleware that seamlessly handles both client and backend concerns would be 🔥#2023-02-2508:47tatutyes, it works great! it was easy to do#2023-02-2508:49tatutI made a system I can call like (call. :some-service-name {:foo 42 :bar 123}) and it will invoke a service multimethod on the server in a wrap, and catch any errors… error is shown on client in an alert#2023-02-2508:49tatutI even added an m/observe that uses NProgress to show a generic top loader line#2023-02-2418:56Dustin GetzDeployment is coming, we have our demos running on fly .io $6/mo tier and performance is great so far, we tested Philadelphia <> Paris, will release the build scripts next week
Rapid scrolling over node_modules from Philly to Paris with great performance for $6, it is a sight to behold#2023-02-2516:58adamfreyI am attempting to use the shadow-css library with electric. Shadow-css has one main macro css which outputs a class string (https://github.com/thheller/shadow-css#using-the-css-macro).
When I try to use that macro and write:
(dom/h1 (dom/props {:class (css {:color "red"})})
(dom/text "minimal todo list"))
I get this error:
(def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Can't take value of macro shadow.css/css
Is that expected behavior at this point? Or is there a different way I could use this library with electric?#2023-02-2517:04Dustin Getztrying to repro#2023-02-2517:06JI like put the styles in a global map as:
(def styles
{:my-component/container (css :flex)
:my-component/title (css :text-gray-800)})
And in electric component:
(dom/h1 (dom/props {:class (styles :my-component/title)}))
This method works perfectly with electric#2023-02-2517:10adamfreytrue, but then I have to give names to everything, which I'd prefer not to be required to do everywhere#2023-02-2517:12Dustin Getzi've reproduced but don't understand it, will need to pass it on to the team#2023-02-2517:15Geoffrey GaillardThis is a bug in the handling of :class .
:class can take a string or a collection of strings.
Here the props macro thinks (css ...) is a list of classes.#2023-02-2517:17adamfreyah, I see. ok, I can workaround for now with J's suggestion until there's a fix. Thanks for looking into this so quickly!#2023-02-2809:00Geoffrey GaillardThis is fixed on latest master#2023-02-2607:41Ben SlessDo you think electric would be a good fit for building something like forum software?
How do logins and permissions work in electric?#2023-02-2607:53tatutafaict, permissions would work like in any web software, your server side calls should only access what the current user is allowed to do#2023-02-2607:58tatutI don’t see why you couldn’t build a form in electric, but… even every reader will need a websocket connection and therefore resources from the server. I would wager that if a forum is publicly browsable, it is much easier to cache unauthenticated pages with a traditional full page server side rendering solution.#2023-02-2608:00tatutI don’t know the internals that well, but it seems it would be difficult to cache the WS messaging in any meaningful way#2023-02-2612:34Dustin Getznot a good fit yet, need SSR and “resumable” rendering, all of which are naturally done from signals based rendering approaches as proven by Solid.js which is dominating SSR benchmarks right now#2023-02-2612:37Dustin Getzhowever even though it can be done, “website” type apps is not our current focus today; Electric today is for ultra dynamic stateful apps like google docs or webflow or IDE#2023-02-2612:41Dustin Getzlike we want to rebuild datascript in electric clojure so we can make the query engine network transparent, slice the indexes so they are partially client side so that “last mile” filtering happens on the client - offline first datascript! that’s ~what~ the kind of thing electric clojure is intended to make possible#2023-02-2612:43Dustin Getzon the other hand, if you tried to build a forum with electric today, the client/server boundary can be refactored and moved around at will, so you could choose to preload the page in one message and then not load anything else until you navigate#2023-02-2612:45Dustin Getzso in theory you could probably make it work, even release the websocket when not needed, or use cloud flare durable objects (edge network for websockets with durable state) - lots of possible ways to scale this#2023-02-2612:48tatutInteresting, is there a way to release and restart the ws? #2023-02-2612:49tatutWould you serialize some representation of the state to ”resume” from?#2023-02-2612:51Ben SlessI've done a bit of hacking on DS, would be happy to lend a hand with that#2023-02-2614:02Dustin GetzNo good way to restart the WS today (it reboots the page if you do, both client and server). We already did a first POC of resyncing state on reconnect, but hit a design flaw in the wire protocol preventing us from doing it efficiently. Maybe next iteration? historically we do a major revision about every 3 months#2023-02-2615:23nakkayaWhat would be the idiomatic way to update a table in a page, without redrawing/reloading the whole page. I have a table in a page, rows are fetched from the db and table is build, I am trying to add delete functionality, a button deletes the item from the db and removes the row.
(dom/tbody
(e/for [[id fname lname dob] (e/server
(->> (db/patients (e/client (user.)))
(map #(vector (str (:patients/id %))
(:patients/first_name %)
(:patients/last_name %)
(str (:patients/date_of_birth %))))))]
(dom/tr
(dom/props {:id id})
(dom/td
(dom/props {:rowspan "1", :colspan "1"})
(dom/text fname))
(dom/td
(dom/props {:rowspan "1", :colspan "1"})
(dom/text lname))
(dom/td
(dom/props {:rowspan "1", :colspan "1"})
(dom/text dob))
(dom/td
(dom/props {:rowspan "1", :colspan "1" :align "right"})
(h/Button. "Delete" "btn btn-sm btn-outline-secondary"
(e/fn [_]
;;(e/server (db/patient-delete id))
;; (-> js/document
;; (.getElementById id)
;; (.remove))
))))))#2023-02-2615:48Dustin GetzDid you see the SQL hello world: https://gist.github.com/dustingetz/1960436eb4044f65ddfcfce3ee0641b7#2023-02-2616:02nakkayaI did see that I had separate question about that I tried to follow that and wrapped the snippet in a try catch and e/wrap the db calls but I got no output no error. So if I understood the example correctly, atom changes which triggers the watch so any code within the scope of that watch is rerun?#2023-02-2617:44Dustin Getzchanging the atom causes the watch to update, which causes any expressions that depend on the watch to update (but not the whole scope, it is fine-grained updates based on a reactivity graph)#2023-02-2617:50Dustin Getz#2023-02-2617:00vincentHi all#2023-02-2617:01vincentI want to make a component that displays the k,v pairs of a map atom as an html table#2023-02-2617:01vincentso far i did#2023-02-2617:03vincent(e/defn AtomTable [map-atom]
(e/client
(dom/div
(dom/table
(e/for [[k v] map-atom]
(dom/tr
(dom/td
(dom/text k v))))))))
#2023-02-2617:15tatutDid you e/watch the atom to get the valua?#2023-02-2617:16vincentum, yes i have that#2023-02-2617:16vincentgood q#2023-02-2617:16vincent#(:clj (defonce !users (atom (list))))
(e/def users (e/server (reverse (e/watch !users))))
#2023-02-2617:17vincentfor examlpe, although want 2 generalize#2023-02-2617:24tatutfwiw, this works for me
(e/defn AtomTable [val]
(e/client
(dom/table
(e/for [[k v] val]
(dom/tr
(dom/td (dom/text k))
(dom/td (dom/text v)))))))
(e/defn Other []
(let [atom-map (atom {:foo 42})
atom-map-value (e/watch atom-map)]
(AtomTable. atom-map-value)))
#2023-02-2617:24tatutI can call AtomTable from another component and it renders ok#2023-02-2617:29Dustin Getzsee https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_3_system_properties.cljc#2023-02-2617:35vincentthanks i had an error in an unrelated swap! that was wrecking the component 😅#2023-02-2617:55vincentactually, not sure :thinking_face: can you define the atom outside the component as clj and still watch it? i'm getting 'n is not a function' :thinking_face:#2023-02-2617:55Dustin Getzyes#2023-02-2618:00vincentok probably just a typo somewhere.#2023-02-2618:00vincentx)#2023-02-2618:00vincentgonna go pour cold water on my head#2023-02-2618:01Dustin Getzmake sure to watch the atom from the same place it is defined at#2023-02-2618:06vincenti define it on the server, then i want the ui component to ppulate a table of it... hmm#2023-02-2618:11xificurCwatch on the server, use the watched value on the client#2023-02-2618:16vincente/for it on the server?#2023-02-2618:16vincentand then pass the resultant table to the client?#2023-02-2618:18vincentoh i see i can destructure on server x)#2023-02-2619:29vincentI guess my question is, can I display all the server atoms to the client in real-time?#2023-02-2619:56Dustin Getzsure#2023-02-2617:03vincentnot sure what i'm missing 😅#2023-02-2618:00vincentgonna go pour cold water on my head#2023-02-2619:29vincentI guess my question is, can I display all the server atoms to the client in real-time?#2023-02-2622:21vincentif i want to output to both js console and clojure sys out can i do so in the same e/defn?#2023-02-2622:29xificurCSure!#2023-02-2622:22vincent(e/client (.log js/console "stuff')) ?#2023-02-2623:20vincentokay i got some of the stuff working, it was very nonobvious what was not right x)#2023-02-2623:50Dustin Getzany ideas how could we improve the tutorials? what document do we need to write that would have gotten you what you need quicker?#2023-03-0201:09vincentno idea. the way i was doing it was wrong and the example was right x)#2023-03-0201:49vincenti don't mind the learning curve.
it is factored into the cost of taking on a new framework always.
in this case, it is very profitable because i only need to learn it once to be productive many times.
i looked at a basic example and went as far as i could writing code myself.#2023-03-0201:51vincentbut when it didn't work as expected, i had to turn to the examples again.
i think a side-by-side page of the code and the demos would be lovely and probably helpful.
a big game changer for me was creating the AtomTable component and being able to live introspect changing values on the page while working / recompiling. a peek into the "db" of the app. i think those sorts of things will help people grok faster.#2023-03-0203:14Dustin Getzdo you have code for AtomTable that we should include in a tutorial? send me some scraps of what you have and i'll think about how to include it#2023-03-0203:15Dustin Getzty for the feedback & ideas, keep it coming#2023-02-2704:00bherrmannfyi: Small electric clojure program to display the unix "fortune" into a web page. https://gist.github.com/bherrmann7/765f4074dfac013527402fa7f4016ee5
This electric clojure is very very cool.#2023-02-2713:30Dustin GetzAll – help Electric by sending me your demos if you haven't already! I am writing a launch recap blog post. If you don't have a video, tell us what you're working on, here in this thread or by DM!#2023-02-2717:40Dustin GetzHeads up: in master we just landed a changeset that stabilizes uncommon shadow-cljs hot code reloading problems. Also it makes #?(:cljs (:require-macros user.demo-1-hello-world)) no longer needed, you don't have to remember to put it at the top of your files anymore. We think the changeset is stable, would love if more people tested it this week and if you have any hot code reloading issues please tell us!#2023-02-2718:36vincentwishlist: built-in analytics#2023-02-2718:41markaddlemanWhat kind of analytics are you thinking? Performance? Usage? Workflow?#2023-02-2721:42nakkayaIs there a gist/doc somewhere that shows how to use history and routing with a little more detail? I am currently using an atom, a watch and a condp to handle routing within the app. Now I would like to store some state in the url so I can support reloading/bookmarking and also get browser back/forward working. I could not figure out how it works from going through the examples in src-dev#2023-02-2721:44Dustin Getzthe hyperfiddle.history module has those features but is highly experimental#2023-02-2721:44Dustin Getzare you looking for a straightforward html5 history integration?#2023-02-2721:57nakkaya> are you looking for a straightforward html5 history integration?
Yes nothing complicated just browser back/forward working. My current implementation is,
(e/def !route #?(:cljs (atom ::patients)))
(e/def route (e/watch !route))
;; then in main view
(condp = route
::patients (Patients.)
::add-patient (AddPatient.)
(NotFound.))
also is there a built in mechanism to set/get URL params so I can keep/retrieve additional information about the route within the URL or should I resort to js.#2023-02-2721:59Dustin GetzTry this: https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc#2023-02-2721:59Dustin GetzThis has not been tested recently, there may be typos#2023-02-2722:00Dustin GetzAlso note this is not public API, we are working on refactoring our git repos so this file may or may not move, just a heads up#2023-02-2722:01Dustin GetzYou can definitely use it#2023-02-2722:03nakkaya👍 thank you#2023-03-0614:37Dustin GetzHey @U5H4U2HEH did you get it working?#2023-03-0614:41nakkaya@U09K620SG I got working a while ago I was gonna post a gist for others but it slipped my mind. Had to modify things a little from the example to get it work but it works as expected.#2023-03-0614:46Dustin GetzWould love to see the gist, will turn it into a tutorial#2023-03-0614:46Dustin GetzFeel free to PR to the main repo if you want the github contributor badge#2023-02-2723:54deniktrying to run electric from source (current master)#2023-02-2723:54denikgetting in cljs
Caused by:
RuntimeException: Can't resolve find-ns
#2023-02-2723:59denik(e/client (dom/div (dom/text "foo"))) ;; works
#_(e/server
(e/client (dom/div (dom/text "foo")))) ;; breaks#2023-02-2805:15denikworks as off this commit#2023-02-2805:15denikcommit 9be03dd66f012971716544aa56c197bf5d0a85b2 (HEAD)
Author: Dustin Getz <<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>>
Date: Thu Feb 23 14:15:51 2023 -0500
electric: efficient clocks moved to core#2023-02-2806:50Geoffrey GaillardWe'll look into it and keep you posted#2023-02-2723:54denikanything besides clojure -T:build compile-java I need to run before it should work?#2023-02-2723:56Dustin Getzthat should not be needed#2023-02-2723:58Dustin Getz@U2DART3HA #2023-02-2806:38Geoffrey GaillardI confirm it should not be needed. Looking into it#2023-02-2801:38JAtkinsIs there a limit to recursive calls? Looks like I caused a stack overflow in the compiler when I tried.#2023-02-2801:51Dustin Getzyou probably ran out of stack, the compiler is not optimized, increase stack size with the jvm flag in our deps.edn#2023-02-2801:52Dustin Getzwhat “recursive call” where you doing exactly?#2023-02-2801:54JAtkinsI'm looking at a recursive reactive call to render a possibly recursive entity tree#2023-02-2801:54JAtkinsrecursive, but terminating, that is#2023-02-2801:56JAtkinsSomething akin to
{:strs-to-render ["...."]
:recursive-target {:strs-to-render ["...."]}}
(e/defn RenderRecursive
[x]
(dom/div
(dom/text (apply str (:strs-to-render x)))
(when-let [r (:recusive-target x)]
(RenderRecursive. r))))
#2023-02-2802:03Dustin Getzi don’t even think we support recursive calls, you have to use a workaround to recur by binding, it’s linked at the bottom of the readme#2023-02-2802:03Dustin Getz(yet)#2023-02-2802:09Dustin Getzbtw when i said you may have run out of stack, i meant the electric compiler ran out of stack during the macroexpansion analysis#2023-02-2802:51JAtkinscan electric functions return functions? e.g. I've got a fn that returns two bits of dom that I'd like to render in different locations. e.g.
(defonce !a #?(:clj (atom 4)
:cljs nil))
(e/defn TestReturnsFn
[text]
{:a (fn []
(dom/div
(dom/text (str ":a TestReturnsFn" text))))
:b (fn []
(dom/div
(dom/text (str ":b TestReturnsFn" text))))})
(e/defn OwningFn
[]
(let [a (e/watch !a)]
(e/client
(dom/div
(dom/text "btn")
(ui/button
(e/fn [] (e/server (e/discard (swap! !a inc))))
(dom/text "btn"))
(let [{:keys [a b]} (TestReturnsFn. a)]
;;(a)
;;(b)
;;(b)
nil
)
))))
------ ERROR -------------------------------------------------------------------
File: /Users/jarrett/code/jra/electric-datomic-browser/src/user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
null
Can't set! local var or non-mutable field at line 8 user.cljs
--------------------------------------------------------------------------------
9 | (app.datomic-browser/DatomicBrowser.)))
10 |
11 | (defonce reactor nil)
12 |
--------------------------------------------------------------------------------
The compiler seems upset though.#2023-02-2806:56Geoffrey GaillardIn your example, :a and :b are regular clojure fns. They should be e/fn since you are using electric-dom inside of them.
e/fn, just like e/defn, is called with new#2023-02-2806:58Geoffrey GaillardThe compiler error message is not helpful, sorry for that. We'll improve it.#2023-02-2811:37Geoffrey GaillardThe error message now looks like this:
Encountered error when macroexpanding hyperfiddle.electric/boot.
`hyperfiddle.electric-dom2/node` is an Electric var and cannot be bound from a Clojure context.
{:file "user_main.cljc", :line 90, :in [(fn* ([] (dom/div (dom/text (str ":a TestReturnsFn" text)))))]}#2023-02-2814:10JAtkinsah, nice. I didn't think about that 👍#2023-02-2820:01Dustin GetzFWIW the language unit tests demonstrate a some cool/tricky things, for example we have https://github.com/hyperfiddle/electric/blob/ed8db1ddece174721b16675267ae96b6be4718ae/test/hyperfiddle/electric_test.cljc#L1818-L1836#2023-02-2820:02Dustin GetzThere is a lot there, depending how deep you wanna go'#2023-02-2809:29tatutis there a good example of testing electric components? do you unit test a component or do browser testing like cypress?#2023-02-2809:33tatutI guess you need a headless browser environment to properly unit test a component#2023-02-2809:36xificurCOur experimental ui4 controls have headless browser tests, e.g. for https://github.com/hyperfiddle/electric/blob/c9c2d8504092d4a034c6f1cb9e85423964e1299c/test/hyperfiddle/electric_ui4_typeahead_test.cljc. These run with every git push. They were written through a cljs repl with a real browser, so you can write/debug the tests incrementally#2023-02-2809:37tatutnice, I’ll check those out#2023-02-2814:18Dustin Getz@U11SJ6Q0K to be clear, are you looking for UI tests? i.e. testing the DOM views?#2023-02-2814:18tatutwell examples of any levels of tests#2023-02-2814:19tatutif you create components that mix frontend and backend stuff, then I would expect to want to test the whole component, not just dom#2023-02-2814:21Dustin Getzhttps://github.com/hyperfiddle/electric/tree/master/src-docs/user/electric shows how to test electric functions, this is an old language tutorial#2023-02-2814:21Dustin Getzhere is our https://github.com/hyperfiddle/electric/blob/master/test/hyperfiddle/electric_test.cljc (we have other lower level tests for compiler/runtime which wont be helpful)#2023-02-2814:22Dustin GetzI am not sure if this will help you though#2023-02-2814:22tatutI’ll look into those, thanks#2023-02-2814:23Dustin GetzIf I was using Electric to build something like Electric Datascript, i'd expect the tests to be mostly unit tests like this#2023-02-2814:23Dustin GetzIf I was using Electric to build a CRUD app, i'd expect the queries to be written in Clojure in the usual way (i.e. this is your problem not an electric problem)#2023-02-2814:24Dustin GetzYour question is thus really about how to test the views of the crud app (the query/dom composition), not the queries in isolation. Do I understand?#2023-02-2814:25tatutthere’s usually a testing pyramid, most tests are unit tests, then there are integration tests that test these in unison and finally e2e tests#2023-02-2814:26tatutso usually the e2e will be cypress/playwright/etc running against a “real” deployed env, but the middle layer of testing that components work together#2023-02-2814:27tatutso, unit tests are covered with normal clojure.test code#2023-02-2814:29Dustin Getzthe middle query/view component integration tests (is this description accurate?) is the part that I dont have a clear answer for you yet — but here's the thing, I have never seen an answer i liked on any stack. What is the best practice for this in a React.js/GraphQL app?#2023-02-2814:30tatutusually there is none, because the frontend and backend is completely separate… I was sort of hoping that with electric there could theoretically be a good solution for this middle level of tests 🙂#2023-02-2814:30Dustin Getzah!#2023-02-2814:31Dustin Getzi agree that with the electric primitives we may be able to do better than the status quo#2023-02-2814:31Dustin Getzi just don't know how yet#2023-02-2814:32tatutthe middle level I have seen is of the “let’s test the APIs we expose to the frontend work properly”#2023-02-2814:32Dustin GetzTo some extent, the point of the JVM is to trust that the JVM has unit tests in place for memory management, so that userland doesn't need to write tests for that anymore#2023-02-2814:33Dustin GetzElectric managed network I think is like that, application programmers shouldn't need to test it#2023-02-2814:34Dustin GetzBut higher level component tests, like "I made a server streamed paginated grid, use CI to prove that it doesn't crash and that it doesn't leak DOM"#2023-02-2814:34Dustin GetzThere is a place for that, and the https://github.com/hyperfiddle/electric/blob/c9c2d8504092d4a034c6f1cb9e85423964e1299c/test/hyperfiddle/electric_ui4_typeahead_test.cljc example Peter linked may be a starting point for developing tests like that. But it's an open question, still working out the idioms#2023-02-2814:36Dustin GetzFor example, our unit tests today don't run a jvm/browser system, they are browser-only or jvm-only and any client/server in the test is loopback#2023-02-2814:36Dustin Getzso we'll need to figure that out#2023-02-2814:38tatutthat would be great, if there was a good solution to that#2023-02-2814:38tatutbut not a deal breaker, usually one needs e2e tests anyway to make sure the final product works in a deployment scenario anyway#2023-02-2814:39tatutbut, if there were a great way to do the middle layer of tests, I would write more of those and less of cypress tests (as I don’t really enjoy writing cypress tests)#2023-02-2814:48Dustin Getzcan i see a snippet of cypress tests that you don't like writing?#2023-02-2814:51tatutcan’t show work code, but even the todomvc test suite https://github.com/tatut/cypress-example-todomvc/blob/master/cypress/integration/app_spec.js (that’s not too bad, but still would prefer clojure)#2023-02-2814:52tatutyou often end up needing to add either classes or some data-cy attributes to your markup to make it more easily testable#2023-02-2814:52tatutand then there’s the dom detach stuff, waits for weird async behaviour#2023-02-2823:19Dustin Getzexcellent link ty#2023-02-2813:02JHi guys! A datascript query should be wrapped into a e/wrap ?#2023-02-2813:07JI have a query who render some items data. Without e/wrap only partial information are re-render (see the right window) when some informations are updated.#2023-02-2813:09JNow if the query is wrap by e/wrap it seems like all the items are re-render when an update occurs:#2023-02-2813:09JThis behaviour is normal?#2023-02-2813:17Geoffrey GaillardThe first video seems to show the right behavior: only what changed is rerendered.
e/wrap is required for IO blocking operations. Since datascript is in-memory, and if your query is fast enough to not block the main thread, then e/wrap is not necessary.
e/wrap will start by throwing Pending, then will emit the result of its body.
In the second video, the list rerenders because of Pending is thrown by e/wrap.
To improve your understanding of Pending, a good exercise would be to add (try … (catch Pending …)) around your e/wrapped query, then move this try block higher up in your code, so you can see the difference.#2023-02-2813:35tatutisn’t the previous value in effect while the query is pending? or is that only when pending is caught#2023-02-2813:57JThanks @U2DART3HA for you explanation. With try/catch , I have the same behaviour. Maybe, I made a mistake. I have something like this:
(e/defn Rooms []
(e/server
(let [rooms (try
(e/wrap (repo/list-room db/db))
(catch Pending _))]
...
(dom/div (dom/props {:class (styles :rooms/room-list-container)})
(e/server
(e/for-by :room/id [{...} rooms]
(e/client ...)))) #2023-02-2814:02Geoffrey Gaillard(catch Pending _) returns nil, so when Pending is thrown, rooms is nil.
Therefore e/for-by will unmount all rows.
When e/wrap returns the query result, rooms is now a collection again, so e/for-by will remount all rows.#2023-02-2814:07Geoffrey Gaillard@U11SJ6Q0K In Electric, try is reactive. You can think about reactive try as a conditional:
• If there is no Pending being thrown
◦ then run the try branch
◦ else also run the catch branch
The main difference between reactive try and reactive if, is that the try branch keeps running when a Pending is thrown.
This way when the Pending exception is not thrown anymore, the catch branch is discarded.#2023-02-2814:47JSo when Pending is thrown, the list will be re-render on each update?#2023-02-2815:13Dustin Getz@J move the try/catch Pending higher. DOM remains attached during Pending (and any other exception). Exceptions do not unmount DOM.#2023-02-2815:26Dustin Getz> isn’t the previous value in effect while the query is pending? or is that only when pending is caught
@U11SJ6Q0K the answer is yes#2023-02-2815:42JI have something like this now:
(e/defn Rooms []
(e/server
(try
(let [rooms (e/wrap (repo/list-room db/db))]
...
(dom/div (dom/props {:class (styles :rooms/room-list-container)})
(e/server
(e/for-by :room/id [{...} rooms]
(e/client ...))
(catch Pending _)))
But still the same behaviour.#2023-02-2815:53Dustin Getz@UHZPYLPU1 we've just now found another regression related to pending, sorry. Not sure if it impacts what you are seeing or not#2023-02-2815:54JNo problem!#2023-02-2821:03Dustin Getz@UHZPYLPU1 the Pending regression is fixed in master#2023-02-2821:06Dustin GetzHere is a demonstration of correct pending behavior, lightly modified from demo-4-chat.
• The document.body flashes yellow due to the Pending thrown at L28, and also L16 (try two tabs)#2023-02-2821:08Dustin Getz@UHZPYLPU1 if you're still seeing the lists rebuild, let's please use a common demo as a starting point, like user.demo-4-webview or user.demo-3-system-properties or user.demo-4-chat (whichever fits your situation best)#2023-03-0106:03JThanks @U09K620SG I will take a look!#2023-03-0112:37J@U09K620SG it works! Thank#2023-02-2821:03Dustin GetzThe pending regression in master is fixed, also increasing confidence that master hot code reloading is stable#2023-03-0100:55tobiasJust wanna say that playing with Electric is a blast and I can't wait until you guys publish instructions for how to build for production (uberjar etc).#2023-03-0101:09Dustin Getzdeploy examples imminent!#2023-03-0103:14vincentHello, I need some help figuring out how to change stuff on click.#2023-03-0103:14vincentI think I want (dom/on ...)#2023-03-0103:14vincentAh okay need to do (e/fn ...)#2023-03-0103:29vincentI want to make clientside views#2023-03-0103:29vincentso like, some clientside atoms that keep current viewname, and then change those on click.#2023-03-0103:31vincentwoot i got it working#2023-03-0104:00vincentwoot#2023-03-0108:43JHi! On master, I have this error Could not initialize class hyperfiddle.electric.impl.env$maybe_load_clj_ns_BANG_$fn__30671 with this code:
(e/def foo)
(e/defn App []
(e/server
(binding [foo "Foo"]
(e/client
(dom/h1 (dom/text "foo"))))))
I miss something?#2023-03-0108:44Geoffrey GaillardThank you for the report. You did not miss something. We are looking into it#2023-03-0111:25Dustin Getzplease post your user.clj, user.cljs and also what commands do you use to start the app?#2023-03-0111:36JHere, the files => https://gist.github.com/jeans11/b0e5dbc68d2ac13623774c55aeac0aa5
To start the app, I launch the main function in the dev namespace.#2023-03-0111:39Dustin Getzwhat happens if you require electric in dev.core#2023-03-0112:34JYes it works now! Thank guys!#2023-03-0113:34NikolaiHello! Awesome project, thanks for doing that! I have tried the "demo-6-two-clocks" and can not figure out why after a few manual page reloads, the server clock (e/server e/system-time-ms) loses its reactivity? The client clock (e/client e/system-time-ms) is going forward, but the server's one is just halted, so I need to restart the clj.#2023-03-0113:47Dustin Getzthat shouldn't happen#2023-03-0113:48Dustin GetzAre there any errors in the server log? What is your OS and system specs?#2023-03-0113:48Dustin GetzHow are you running the demos?#2023-03-0115:06NikolaiNo, that is happening without any errors. I just compile and run from the master branch of Electric using the VSCode terminal. I am running it on Apple M1 macOS Ventura 13.2.1. Clojure CLI version 1.11.1.1224, OpenJDK Runtime Environment Temurin-17.0.6+10#2023-03-0115:30Dustin GetzCan you try a non-Temurin JDK#2023-03-0116:48NikolaiSure, thanks I will try. And sorry, there is an error after I reload a page. Here is the log:#2023-03-0117:29Dustin GetzAre you using firefox?#2023-03-0118:20Dustin GetzI saw that error in a user call once from Firefox – the error comes from the prior page instance and somehow leaks across a hard refresh into the new page#2023-03-0118:20Dustin Getzthe error simply means that the websocket connection was killed#2023-03-0118:21Dustin GetzTry Chrome to confirm this#2023-03-0120:45NikolaiYes, I tried it exactly in Chrome (version 110 - arm64) and it gives me that error on page refresh. As you suggested I'll try a non-Temurin install.#2023-03-0121:19NikolaiI have run master branch with non-Temurin java version "17.0.6" 2023-01-17 LTS and still get the same behaviour and error message. In Safari browser it also halts after 3-4 page refreshes, but with no logging of the error.#2023-03-0121:20Dustin Getzhey#2023-03-0121:20Dustin Getzthinking#2023-03-0121:20Dustin GetzThere's a java tool to inspect threads and detect deadlocks. Do you happen to know how to use it?#2023-03-0121:22Dustin Getz"VisualVM"#2023-03-0121:25Dustin Getzoh hey i reproduced it#2023-03-0121:26NikolaiOh, I have no experience in using Java dev tools (new to me). Mostly I programming Smalltalk and JavaScript. I just wonder if someone else experience the same as me?#2023-03-0121:26Dustin Getzthanks for the report!#2023-03-0121:27Dustin Getz#2023-03-0121:31Dustin GetzWe'll get it fixed, the issue is almost certainly constrained to the two clocks demo#2023-03-0121:35NikolaiWill looking forward to this, am very excited with Electric, Thanks!#2023-03-0421:19Dustin Getz@U04RB45M8J3 we investigated; the deadlock is an interaction with hard refresh (which tears down the system and the teardown can deadlock with the sleep scheduler). If you don't refresh the two clocks page you won't get a deadlock.#2023-03-0421:20Dustin GetzWe'll fix it obviously but we've backlogged it because it's not something that happens in real world circumstances#2023-03-0421:21Dustin Getz(This is a Missionary bug btw, in https://github.com/leonoel/missionary/blob/master/java/missionary/impl/Sleep.java, its very localized)#2023-03-0421:24Dustin GetzAlso, if necessary we can slow down the timer from 1000hz to 100hz which will dramatically lower the probability (increasing the speed of the timer recently is what made the problem surface)#2023-03-0423:29NikolaiDustin, thank you very much for letting know and clarifying things! Yes, this is not so critical and does not prevent making very fantastic things with Electric just right now.#2023-03-0117:32coetry@dustingetz and team, just wanted to thank you guys for releasing this to the world. The hype is warranted and i’m still wrapping my head around the implications. Even React is converging into co-located server and client components, but having access to that paradigm within the context of Clojure and a single unifying layer / ecosystem protects us from much fragmentation at the org / human resource level and helps us really get into deeper flow states 🚀 . Still hacking around, demos on the way!#2023-03-0117:33coetrygood read to keep up with where the React world is headed: https://beta.nextjs.org/docs/rendering/server-and-client-components#2023-03-0118:25Dustin GetzThank you Allen, appreciate the kind words! any interest in a zoom onboarding?#2023-03-0119:03coetryyeah 100%#2023-03-0119:03coetrylmk details#2023-03-0118:37Dustin GetzSVG is in master
example code: https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_svg.cljc
implementation is https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_svg.cljc#L8-L19
(plus a 10 line improvement to electric-dom to support DOM attribute namespaces)#2023-03-0218:47Dag NorbergHi, how can I use this in electric-starter-app? The latest version on Clojars seems to be from Feb 19?#2023-03-0219:12Dustin Getz@U04P1F17B2N you can do {:git/url "" :git/sha "...."} in the deps.edn#2023-03-0219:13Dustin Getzi need to do a proper release, it's coming - not actually sure if there are any breaking changes#2023-03-0219:18Dag NorbergThanks! I guess the dependency should be electric and not electric-starter-app though?#2023-03-0219:27Dag NorbergHave updated the electric-clojure-painter repo now, SVG works beautifully 🙂#2023-03-0118:59Michael WIn the new master e/wrap is marked as deprecated? What to use for blocking calls now?#2023-03-0119:01Dustin Getze/offload ; please migrate (e/wrap (teeshirt-orders db search)) to (e/offload #(teeshirt-orders db search))#2023-03-0119:01Dustin Getznote the ##2023-03-0119:01Dustin GetzWe'll release a proper versioned artifact soon and provide a changelog#2023-03-0120:00tatutThat feels like a better name that suggests what it does#2023-03-0120:45Dustin Getzyeah, thanks#2023-03-0200:57JAtkinsLive reloading seems to break sometimes. I'm in the electric datomic browser demo adding a new router endpoint, and when I make a new file anything in that file stops updating.
(ns app.trial
"must have datomic on classpath, and must load 'test ns"
;;#?(:cljs (:require-macros app.trial)) ;; I'm using a git dep on latest electric, no longer needed
(:require
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]))
(e/defn Child
[]
(e/client
(dom/div (dom/text "hirt"))) ;; Changing the string has no effect on the browser
)
;; In the router in app.datomic-browser
(e/defn Child
[]
(e/client
(dom/div (dom/text "hidb"))) ;; Changing the string updates the browser
)
;; Referring to the second file, doesn't update properly
::test-page (history/router 1 (e/server (app.trial/Child.)))
;; Referring to this file, updates properly
::test-page (history/router 1 (e/server (Child.)))
This seems to be isolated to the datomic browser repo, since demos in the electric repo work fine as well. Any ideas?#2023-03-0203:09Dustin GetzOk, from your comment it seems you have this linked up against electric master, so you know about and have the latest hot code reloading changes#2023-03-0203:11Dustin GetzAh, the :dev/always flag might be on the wrong file now ... checking#2023-03-0203:12Dustin Getzthis looks like a typo on our part: https://github.com/hyperfiddle/electric-datomic-browser/blob/main/src/user.cljs#L1#2023-03-0203:13Dustin Getzcompare to https://github.com/hyperfiddle/electric/blob/master/src-dev/user.cljs#L1#2023-03-0203:13Dustin Getzthe meta comes first#2023-03-0203:21JAtkinsAh! Ok perfect thanks! I didn’t even realize that was in the ns decl :)#2023-03-0201:34vincenti expect the view to update w/ the change in atom state, but some components are not hiding ... hmm#2023-03-0201:34vincentoh oh nvm#2023-03-0201:34vincenti need to change something about how i store user views x)#2023-03-0201:35vincentthe view the current user is experiencing should live on the server, probably tied to the session var#2023-03-0201:35vincentmethinks.#2023-03-0210:05JHi guys, I have a weird behaviour with datalevin and pending. Sometimes, the catch branch is not unmount. Here, a code to reproduce (https://gist.github.com/jeans11/3401cc6b23f0dd8f889971f6aa76fcac).#2023-03-0211:28Dustin Getzty, we will investigate#2023-03-0214:53xificurCHi @UHZPYLPU1!
Thanks for the repro! https://gist.github.com/xificurC/1cbc281dd49e7eb3c9f5adcba6382158. I'm not sure why your version glitches, but it's a tricky beast that catches Pending on the server and requests the client to render something, which will throw a new Pending!
A better way to do this is to try/catch on the client, which resolves this issue#2023-03-0215:04JThanks @U09FL65DK! You rock#2023-03-0211:21Dave MaysReally excited about Electric, and the demos are helpful.
I apologize for asking this because it is so basic, but are there any walkthroughs or example repos on how to connect to a persistent DB?
Being new to the Clojure world (let alone Electric) I'm still figuring out the DB options (Datomic, Datascript, XTDB) but these Datalog based DBs are part of what draw me to Clojure in the first place, and am leaning towards XTDB (not asking for XTDB specific instructions.)
The standalone starter app's require statement seems to show a conditional #?(:clj allowing us to say what runs on the server side. I was suprised datascript is server side, I thought it was client side for some reason, and "meant to run in the browser." Is datascript appropriate for persistence in production, or would one of the others be needed?
With Electric being automatically "multiplayer", what tells other user's UIs to update? When a bunch of different client UIs need to, update do they each re-read directly from the database, or does a websocket just tell them what pieces to update?#2023-03-0211:24Dustin Getzthere is a rudimentary SQL tutorial linked in the readme, more robust examples coming soon. we use datascript in examples because it is zero setup but conceptually any db works mostly the same#2023-03-1115:00JMaybe portal#2023-03-1115:19Michael WI use portal but never had to interact with it's api. Reveal is another to look at as well.#2023-03-1118:51lread@U7RJTCH6J is doing some interesting experiments in this area too. There is a https://youtu.be/yPurwuTOHCE where he demos to @U6VPZS1EK.#2023-03-1119:11phronmophobicThe tool I demo works best if you already have the data locally. I use it frequently, but it's currently alpha.
You might also interested in https://github.com/wilkerlucio/pathom-viz/. https://www.youtube.com/watch?v=YaHiff2vZ_o#2023-03-1119:12phronmophobicI think #clerk would also work well as a means for exploring an API like github's.#2023-03-1119:20phronmophobicFor exploring github, you could probably do something cool with https://unpkg.com/browse/@octokit/openapi@10.0.0/generated/api.github.com.json#2023-03-1122:26Dustin Getzthank you @U7RJTCH6J i need some time to digest#2023-03-1115:24Michael WI'm having an issue with an svg, one of the properties for an svg animation is called xlink:href, but I cannot seem to make it work using svg/animate
Here is a sample:
(svg/animate (dom/props {:attributeName "font-size"
:from "50"
:to "75"
:dur "1s"
:repeatcount "1"
:begin "click"
:xlink:href (str "#" tilename)}))
This obviously gives me an error in clojure, seeing as that is not a proper keyword. I looked at the macro that creates the animation document element but am not understanding how to translate that property name to a proper clojure keyword.#2023-03-1115:28Dustin Getztry a string#2023-03-1115:31Michael WFailed to execute 'setAttributeNS' on 'Element': '' is an invalid namespace for attributes.
in ( hyperfiddle.electric-dom2/set-property! #object[SVGAnimateElement [object SVGAnimateElement]] xlink:href #tile0 )
#2023-03-1115:39Michael WI tried "xlink:href" and got that error, when I try :xlink/href that gets translated to just href on the properties, not "xlink:href" which would be the correct property.#2023-03-1116:40Dustin GetzOk, I will take a look with the team next week, are you blocked?#2023-03-1116:42Michael WNo it's just animations, I've got plenty of other stuff that is higher priority, I was just messing with animations for fun. Eventually I would like to get the animations working, but it's not affecting functionality of my app at all.#2023-03-1309:21Geoffrey GaillardA fix is available on latest master.
Note https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/xlink:href, but might be needed for retrocompat.#2023-03-1118:12Michael WJust a small thing, not sure it's even worth a fix. I noticed that if I have an atom that is a dictionary, and a watch setup on that atom, I cannot use the watch as a function, like I could with a "real" dictionary.
#?(:clj (defonce !state (atom {:test true})) :cljs nil)
(e/def state (e/server (e/watch !state)))
(e/client (dom/p (dom/text (str (state :test))))) => fails with an error about state not being a function
(e/client (dom/p (dom/text (str (:test state))))) => succeeds
(e/client (dom/p (dom/text (str (get state :test))))) => succeeds
#2023-03-1217:48Dustin Getzlogged a bug, i think it's just an oversight#2023-03-1218:30Dustin GetzElectric XTDB starter app is repaired: https://github.com/hyperfiddle/electric-xtdb-starter#2023-03-1218:32Dustin Getz@UAB2NMK25 this fixes the protocol reloading issues you saw. You should be able to just upgrade your app to electric v2-167#2023-03-1218:33Dustin Getzbtw we haven't officially released v2-167 yet, there is a breaking change related to resource classpath, you'll need to perform https://github.com/hyperfiddle/electric-starter-app/commit/3ab72a62d3be66b2567e7ad3c1a88a21db409fc2 on your deps.edn#2023-03-1219:33Michael WSweet, thanks for the heads up, I'll play with this a bit later today or tomorrow night.#2023-03-1308:04Canhello, can I ask something? The calculation codes are working as I wanted. But I want to change that code more simply and readable. So I defined a function to call calculations. But when I call the function but that doesn't work. Why? Can anyone help me?#2023-03-1308:12Geoffrey GaillardPrefix VS Infix notation.
In, say, python you’d write s * w / h * h – *in*fix
In Clojure: (/ (* s w) (* h h)) – *pre*fix
See, in Python / and * are operators. And they have the famous operator precedence rules (PEMDAS).
s * w / h * h works because multiplication has precedence over division. This is hard to remember and get right.
Lisps (like Clojure) are simpler:
• / and * are just functions
• The first elements in a list ( … ) is considered a function to be called. -> no need for operator precedence.#2023-03-1309:01CanOps that's a very basic mistake, thank you so much. Still my brain works like java python-dev 🙂#2023-03-1313:28Dustin GetzPlease call (e/watch ...) only once per atom and share the return value. Otherwise, each individual watch will trigger separately, causing extra wasteful computation and also causing your app to see inconsistent states ("reaction glitch")#2023-03-1313:34Dustin GetzThe same applies to swap!, it is better to modify the atom "as a transaction", like this:
(def !state (atom {}))
; bad - each swap! will cause your app to update
(swap! !state assoc :a 1)
(swap! !state assoc :b 2)
(swap! !state assoc :c 3)
; good - one transaction = one update
(swap! !state assoc :a 1 :b 2 :c 3)
(swap! !state #(merge % {:a 1 :b 2 :c 3}))
(swap! !state merge {:a 1 :b 2 :c 3})
(swap! !state #(-> %
(assoc :a 1) ; composable updates
(assoc :b 2)
(assoc :c 3)))#2023-03-1316:02Dustin GetzBreaking change in master: hf/*http-request* is now e/*http-request*, https://github.com/hyperfiddle/electric/commit/9f6f1679bd0fd14761424cfbd4e2a65b61cced61#diff-e2cf315a7c2fe086e9d235b2596d161efa8e91722ea6b2ed7f3abeaf9db81587#2023-03-1412:36Fit Openehello, I just started learning Clojure and I really appreciate this Electric library. However, I don't know how to use it in an Expo + React Native project. I tried modifying the https://github.com/PEZ/rn-rf-shadow project, but I've been struggling for a day with configuring deps.edn and shadow-cljs.edn, and I can't seem to make it work. Could you give me some advice on this? Thank you.#2023-03-1412:42Dustin GetzWe have not yet tried React Native, it will likely not work yet with Electric#2023-03-1412:43Dustin GetzI have no reason to believe it won't work, but we know nothing about React Native as a platform, so I'd assume some minor work is required#2023-03-1412:50Fit OpeneThank you for your reply, I will keep trying and hopefully make some progress soon.#2023-03-1418:58denikwould like to set pending state on an atom for a loading state but strangely pending is caught twice
(defmacro set-pending [atm & body]
`(try (let [ret# (do ~@body)]
(and (do ret# true)
(do (println :done) true)
(reset! ~atm false)
ret#))
(catch Pending e#
(println :pending)
(reset! ~atm true)
(throw e#))))#2023-03-1419:28xificurCis this on master? There were a couple of fixes around try/catch lately#2023-03-1419:29denikyup I’m on master#2023-03-1419:29xificurCit's also possible body is throwing Pending twice. Can you provide a repro?#2023-03-1419:30denikeven if it does that should be okay, is there an ultimate Success / return state?#2023-03-1419:32xificurCI mean if the body goes e.g. Pending, v, Pending, v then seeing 2 Pendings is fine, right?#2023-03-1419:35xificurCthis prints a single pending (p/run (set-pending (atom 1) (p/offload #(do 2))))#2023-03-1419:41denik#2023-03-1419:41denikthis sets pending to true but not back to false#2023-03-1419:43xificurCI see. That's because in electric almost everything is running concurrently. (try a b c) will run all of them at the same time. So (reset! !p false) will finish before hitting the catch handler#2023-03-1419:44xificurCYou can do (catch Pending _ (reset! !p true) (e/on-unmount #(reset! !p false)))#2023-03-1419:50xificurCto elaborate a bit, the electric code
(do (throw (ex-info "boom" {}))
(println "hi"))
will both throw and print, because all of dos children are running concurrently#2023-03-1420:03denikokay great thanks that worked#2023-03-1420:04denik(defmacro set-pending [atm & body]
`(try (do
(e/on-unmount #(reset! ~atm false))
~@body)
(catch Pending e#
(reset! ~atm true)
(throw e#))))#2023-03-1420:08xificurCI'd put the unmount call in the catch block. In your use case ( (dom/on "click" (e/fn [_] <here>)) ) it makes no difference though#2023-03-1420:18denikgetting a lot of these after reloads, never before#2023-03-1420:18denikanyone else have that issue?#2023-03-1420:19Dustin Getzi think we are seeing it too#2023-03-1420:19Dustin Getzwhat commit are you on?#2023-03-1420:36denik7d184721c4b470f18002fbc248dc3018329526c7#2023-03-1420:36denikhave been seeing it for a while though#2023-03-1420:37Dustin Getzare you doing css work lately?#2023-03-1420:42denikyes though it’s been intermeshed w everything else#2023-03-1420:55Dustin GetzWe're bisecting, the key piece of the repro is to touch a css file and ALSO load it like this: (dom/link (dom/props {:rel :stylesheet, :href "/todomvc.css"}))#2023-03-1420:55Dustin GetzSeems we can repro all the way back to last summer#2023-03-1420:55Dustin GetzAre you also loading css like that?#2023-03-1421:09adamfreyis there a different / better way to load a css file for electric?
That form matches https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L57#2023-03-1421:11xificurCthis is buggy, we're searching why. Seems to be a months old regression#2023-03-1421:12Dustin GetzYeah you can probably work around this by loading css through index.html#2023-03-1421:12Dustin GetzPeter bisected this back to August 2022, this is the first time we noticed#2023-03-1510:38xificurC@U050CJFRU this might be related to a fix we just landed, can you try updating to latest master and see if the problem still persists?#2023-03-1516:30denik@U09FL65DK it seems to be gone! will report back should I see it again. thank you!#2023-03-1717:12denik@U09FL65DK on master still seeing this occasionally#2023-03-1717:13denik#2023-03-1807:25xificurCSorry to hear that. We haven't seen this after the latest fixes, so if you find a good repro let us please know#2023-03-1500:29Michael WI upgraded to the latest master sha and am now seeing errors like this for portal:
The required JS dependency "http" is not available, it was required by "portal/runtime/node/launcher.cljs".
Dependency Trace:
user.cljs
app/tic_tac_toe.cljc
portal/api.cljc
portal/runtime/node/launcher.cljs
Searched for npm packages in:
/home/mwhitford/clojure/electric-ttt/node_modules
http is part of the node-libs-browser polyfill package to provide node-native package support
for none-node builds. You should install shadow-cljs in your project to provide that dependency.
npm install --save-dev shadow-cljs
I did not change anything, and yarn shows shadow-cljs in devDependencies and installed.#2023-03-1502:05Michael WI was able to get this working again by adding the following dep to deps.edn:
http-kit/http-kit {:mvn/version "2.7.0-alpha4"}
Not sure why it stopped loading it as a dep from portal, it's in the deps.edn for portal....#2023-03-1502:06Michael WSpoke too soon, that cleared the issue when loading a file into the repl, but the electric compile is still failing with the same above error.#2023-03-1502:46Dustin Getzwe will take a look in the morning. i think we removed our dependency on npm’s shadow entirely, the error (from portal?) suggests that perhaps shadow-cljs must be listed as a normal dependency in node modules? i think this is a portal question#2023-03-1502:47Dustin Getzscan the last 20 commits on electric github and you might see we removed shadow from package.json or similar#2023-03-1502:47Michael WI'll look thanks for the pointer.#2023-03-1514:12markaddlemanI think the latest portal release is broken. https://clojurians.slack.com/archives/C0185BFLLSE/p1678861984692559?thread_ts=1678861361.210829&cid=C0185BFLLSE#2023-03-1517:37Yab MasIm running into an issue while trying to add the electric-ws-handler to my own server-setup. I wouldn't be suprised if Im doing something basic wrong, but the error seems to be inline with what I see in the source-code but I dont fully understand it so Im posting here.
Im not concerned with java 8 compatibility so upgraded to jetty9-adapter and basically followed the example in the electric-jetty-server ns.
When I load the page in the browser it seems to crash on-connect with the followin error: Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
With the last bit of the trace being:
[clojure.lang.AFn throwArity AFn.java 429]
[clojure.lang.AFn invoke AFn.java 36]
[hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_connect__23415 invoke electric_jetty_adapter.clj 57]
Looking at the source-code of electric-ws-adapter I indeed see that handler-f is called with 2 args and electric-ws-message-handler expects 3, but I rreally dont understand most of the code there.
Any clue whats going on? I'll post source-code and client/server errors in comments#2023-03-1517:38Yab Mas(defn wrap-websocket [next-handler ws-handler]
(fn [request]
(if (jetty/ws-upgrade-request? request)
(jetty/ws-upgrade-response ws-handler)
(next-handler request))))
(defn app [{:keys [routes
custom-content-negotiation
ws-handler
jira-handlers]}]
(let [content-negotiation (muuntaja-options custom-content-negotiation)]
(->
(ring/ring-handler
(ring/router
[routes
(atlassian-connect/routes jira-handlers)])
(constantly not-found-response))
(middleware/wrap-format content-negotiation)
(catch-req-middleware)
(wrap-params)
(wrap-resource "public")
(wrap-content-type)
(wrap-index-as-root)
(wrap-websocket ws-handler))))
(defn start-server
[{:keys [port]
:as server-config}]
(println (str "\nStarting server on port: " port "\n"))
(jetty/run-jetty
(app server-config)
{:port port
:join? false}))#2023-03-1517:39Yab MasFrom different ns:
(def server-config
{:port (server/get-port "electric")
:ws-handler (adapter/electric-ws-adapter adapter/electric-ws-message-handler)})
(defn start-server [] (server/start-server server-config))
#2023-03-1517:42Yab Mas#2023-03-1517:43Yab MasERROR hyperfiddle.electric-jetty-adapter: Websocket error #error {
:cause Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
:via
[{:type org.eclipse.jetty.websocket.api.exceptions.CloseException
:message org.eclipse.jetty.websocket.core.exception.WebSocketException: WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN method error: Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
:at [org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler convertCause JettyWebSocketFrameHandler.java 506]}
{:type org.eclipse.jetty.websocket.core.exception.CloseException
:message org.eclipse.jetty.websocket.core.exception.WebSocketException: WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN method error: Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
:at [org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession lambda$onOpen$5 WebSocketCoreSession.java 404]}
{:type org.eclipse.jetty.websocket.core.exception.WebSocketException
:message WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN method error: Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
:at [org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler onOpen JettyWebSocketFrameHandler.java 187]}
{:type clojure.lang.ArityException
:message Wrong number of args (2) passed to: hyperfiddle.electric-jetty-adapter/electric-ws-message-handler
:at [clojure.lang.AFn throwArity AFn.java 429]}]
:trace
[[clojure.lang.AFn throwArity AFn.java 429]
[clojure.lang.AFn invoke AFn.java 36]
[hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_connect__23415 invoke electric_jetty_adapter.clj 57]
[ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__2484 invoke websocket.clj 153]
[ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6 onWebSocketConnect nil -1]
[org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler onOpen JettyWebSocketFrameHandler.java 177]
[org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession lambda$onOpen$6 WebSocketCoreSession.java 410]
[org.eclipse.jetty.server.handler.ContextHandler handle ContextHandler.java 1463]
[org.eclipse.jetty.server.handler.ContextHandler handle ContextHandler.java 1482]
[org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker$1 handle AbstractHandshaker.java 212]
[org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession onOpen WebSocketCoreSession.java 410]
[org.eclipse.jetty.websocket.core.internal.WebSocketConnection onOpen WebSocketConnection.java 542]
[org.eclipse.jetty.io.AbstractEndPoint upgrade AbstractEndPoint.java 451]
[org.eclipse.jetty.server.HttpConnection upgrade HttpConnection.java 419]
[org.eclipse.jetty.server.HttpConnection onCompleted HttpConnection.java 450]
[org.eclipse.jetty.server.HttpChannel onCompleted HttpChannel.java 968]
[org.eclipse.jetty.server.HttpChannel handle HttpChannel.java 485]
[org.eclipse.jetty.server.HttpConnection onFillable HttpConnection.java 282]
[org.eclipse.jetty.io.AbstractConnection$ReadCallback succeeded AbstractConnection.java 319]
[org.eclipse.jetty.io.FillInterest fillable FillInterest.java 100]
[org.eclipse.jetty.io.SelectableChannelEndPoint$1 run SelectableChannelEndPoint.java 53]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy runTask AdaptiveExecutionStrategy.java 412]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy consumeTask AdaptiveExecutionStrategy.java 381]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy tryProduce AdaptiveExecutionStrategy.java 268]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy lambda$new$0 AdaptiveExecutionStrategy.java 138]
[org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread run ReservedThreadExecutor.java 407]
[org.eclipse.jetty.util.thread.QueuedThreadPool runJob QueuedThreadPool.java 894]
[org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run QueuedThreadPool.java 1038]
[java.lang.Thread run Thread.java 833]]}#2023-03-1517:49Dustin GetzCan you use the Jetty 9 version that the starter app uses? (Is there a reason for upgrading to Jetty 10?)#2023-03-1517:50Dustin GetzWe are reorganizing this stuff this week for the next release, we might add CI/CD for the Jetty 10 version#2023-03-1517:53Yab MasNah no reason to use 10, it just looked simpler and I interpreted the comments as if thats the preferred way but you go with 9 for java 8 compatibility. I'll give it a try after dinner!#2023-03-1517:54Geoffrey Gaillardelectric-ws-message-handler takes the ring request as first argument.
You can try with
(defn wrap-websocket [next-handler ws-handler]
(fn [request]
(if (jetty/ws-upgrade-request? request)
(jetty/ws-upgrade-response (adapter/electric-ws-adapter (partial adapter/electric-ws-message-handler request))
(next-handler request))))
#2023-03-1517:56Dustin Getzhe has that#2023-03-1517:56Dustin Getzi don't like this line:
ws-handler (adapter/electric-ws-adapter adapter/electric-ws-message-handler)
#2023-03-1517:57Dustin GetzI don't see a line like that in our example#2023-03-1517:57Dustin Getzand i agree it doesn't typecheck#2023-03-1519:19Yab MasAdding the req with partial like geoffrey suggested worked!#2023-03-1519:20Yab MasI honestely do think what i posted is what you have in the examples, you just dont have that exact line like that because I sent it as an arg. Anyway thanks for the fix!#2023-03-1517:58Michael WOn the latest master after removing the shadow-cljs stuff as shown in the commits on the master branch, all my tap> calls are not being registered. If I tap from the nrepl it shows up both in the shadow-cljs web dashboard and in portal. If I tap from electric code I see nothing. Also on the newest version I am seeing these log entries that I did not before, I don't know if it's related or not, my app is functioning the same.
INFO : Unserializable reference transfer: datascript.lru$cache$reify__51539 datascript.lru$cache$reify__51539@38da68e3
INFO : Unserializable reference transfer: datascript.lru$cache$reify__51539 datascript.lru$cache$reify__51539@3971f8cf
INFO : Unserializable reference transfer: clojure.lang.Atom clojure.lang.Atom@2e08353f
INFO : Unserializable reference transfer: clojure.lang.Atom clojure.lang.Atom@2d13c329
INFO : Unserializable reference transfer: datascript.db.Datom #datascript/Datom [1 :player/session-id "RVgyUGZsRGJ2UEN1dzRGTldRRHd0Zz09" 536870913 true]
#2023-03-1517:59Dustin Getzit looks like an e/server got dropped in a refactor#2023-03-1518:00Dustin Getzlook for places where you're calling into datascript from the client#2023-03-1518:01Dustin Getzthis would also explain the tap> malfunction, if it is called from the wrong site#2023-03-1518:03Michael WSo I just found the serializable issue, I was accessing datascript from the client side where it should have been server side, so the issues with the log entry are fixed. Still not getting taps though, in (e/client) or (e/server) code, taps just do not show up.#2023-03-1518:03Michael WIf I hit tap from a comment, loaded directly to the nrepl it works, but anything in electric code is just being swallowed.#2023-03-1518:04Dustin GetzDoes println work?#2023-03-1518:07Michael Wyeah from server and client println is working, I see it on the terminal for server and on the browser console for client#2023-03-1518:07Dustin Getzalright, if you push your repo i'll take a look#2023-03-1518:08Michael WIt'll be a bit later tonight can only work on this outside work hours and my lunch hour is up.#2023-03-1615:21Michael WI'm not sure exactly what caused this but it just started working for me after a clean start. Been a few days since I closed the repl I guess.#2023-03-1615:30Dustin Getzhm ok#2023-03-1615:30Dustin Getzi suspect an issue on portal side#2023-03-1615:31Michael WI switched to using reveal, portal does not like the new electric seutp.#2023-03-1615:32Michael WIt's just to see taps so whatever works is fine by me.#2023-03-1615:32Michael WBut the problem I was seeing with taps not showing up is gone. It was happening for the shadow web tap stream too.#2023-03-1615:32Michael WNow shadow tap stream and reveal both see taps#2023-03-1522:31Adrian SmithI was trying this demo out earlier today:
https://gist.github.com/dustingetz/d58a6134be310e05307ca0b586c30947
but wasn't sure what to make of the error:#2023-03-1523:50Dustin Getzthe gists aren’t maintained, there is a repo for the painter demo linked from the readme i believe#2023-03-1523:53Dustin Getzmaybe the error is a missing e/client?#2023-03-1609:30Adrian SmithI think the gist is linked from the main readme - probably how I found it#2023-03-1610:00Dag NorbergThe repo is here 🙂 https://github.com/formicagreen/electric-clojure-painter#2023-03-1610:00Dag NorbergLive demo: https://painter.up.railway.app/#2023-03-1611:37Dustin GetzCool! @U04P1F17B2N we will take a look at optimizing that (on my machine the pen lags). It should 100% be fast#2023-03-1612:51Dag NorbergYeah I managed to fix the cursor performance using only client-side values. Tried doing the same with the “currently drawing path” but somehow it didn’t work#2023-03-1610:41JHi guys! There is a reason to not pass the event here https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_ui4.cljc#L99?
Because when a button is inside a form sometimes we want to preventDefault the event.#2023-03-1610:49Geoffrey GaillardThis is a good question. We’ll discuss it.
In the meantime, <button role="button">…</button> will not submit your form on click (default role for a form button is submit).#2023-03-1610:51JThanks @U2DART3HA 👍#2023-03-1611:58Dustin Getz@UHZPYLPU1 also you can use the lower dom/button + dom/on interface, example is https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_chat.cljc#L25#2023-03-1612:02Dustin GetzThe reasoning today (which can change) is: ui/input is a high level input that gives you the value not the event. ui/button is supposed to also be high level like that. Interacting with a dom/form would be a lower level concern in this worldview.#2023-03-1615:12markaddlemanI’m curious about incorporating electric into a serverless architecture. I think the big question is how to externalize the server-side memoize caches to something like Redis? Of course, there are other issues like being at to associate a websocket with a stable client id but these are outside electric’s scope and have googleable solutions.#2023-03-1615:28Dustin GetzYou want server state to be durable across restarts? Memo state is stored in Missionary, so we'd likely move the missionary heap into something durable as you suggest
or are you thinking about something else?
Today the server runtime state can theoretically be reconstructed from the client runtime state, so this may not actually be an issue. We POC'ed this already, there's a blocker, it will land after the next missionary design lands#2023-03-1615:30markaddlemanThe first one: I am thinking of durable server-side state across restarts.
Separately, I have a question related to recreating server-side state from the client#2023-03-1615:34markaddlemanUnrelated to servlerless electric, I’m curious about the performance issues around a single client-side atom for its database and a server-side watch on that atom.
Will electric stream all updates to the client-side atom? Or only those changes that server cares about? For example, suppose the atom contains
{:client/cursor-location "blah blah", :shared/key "key"}
The server contains functions like (get db :shared/key) but never reference :client/cursor-location#2023-03-1615:36xificurCa server cannot watch a client atom#2023-03-1615:37markaddlemanoh, well that answers that 🙂#2023-03-1615:37Dustin Getzonly scope (e.g. lexical/dynamic/global/local) is streamed, as reified in the DAG#2023-03-1615:40markaddlemanah, i think i got it. so, if the server could watch a client atom, then update to :client/cursor-location in the atom would trigger a recalc but since none of the server’s scope is changed, no application code would be executed#2023-03-1615:44Dustin Getzany change to an atom will fire watches (this is how clojure atom watches work - see core/add-watch and core/remove-watch). what happens next depends on what your cursor does. From electric's perspective, at any point if a reactive value is the same as it was before, we skip the pointless recomputation and use the memoized value. So any work downstream of that would be skipped.#2023-03-1615:44Dustin Getzdoes that answer your question?#2023-03-1615:45markaddlemanYes, it does.#2023-03-1615:45markaddlemanNow I’m curious why the server cannot watch a client-side atom.#2023-03-1615:46Dustin Getzyou can't move/serialize references#2023-03-1615:47markaddlemanI don’t follow but I think I should re-ground myself in the code before continuing my fanciful architecture ideas 🙂#2023-03-1615:48Dustin Getzyou can do this
#?(:cljs (def !x (atom {:deep {:thing 1}})))
(e/server
(let [x (e/client (e/watch !x))]
(println x))) ; entire value was transferred#2023-03-1615:49Dustin Getzbut !x cannot transfer because it can't be serialized, and e/watch needs to attach directly to the reference, so it needs to be co-located#2023-03-1615:50markaddlemanI understand that part. But suppose the code was this:
#?(:cljs (def !x (atom {:client-only {:thing 1}
:shared {:something "blah"}})))
(e/server
(let [{shared :shared} (e/client (e/watch !x))]
(println shared)))
#2023-03-1615:52Dustin Getze/watch returns a reactive value so that’s allowed#2023-03-1615:52markaddlemanIf I understand correctly, if the client executes (swap! !x update :client-only f)then the bytes will flow across the websocket to the server and electric/missionary would run enough to determine that the server state hasn’t changed#2023-03-1615:52Dustin Getzwe aren’t that smart, the destructuring here happens on the server#2023-03-1615:53markaddlemanSo, there is an opportunity for optimization but it is not currently implemented#2023-03-1615:54markaddlemanI imagine a poor man’s solution might be
#?(:cljs (def !x (atom {:client-only {:thing 1}
:shared {:something "blah"}})))
(e/server
(let [shared (e/client (e/watch-in !x [:shared]))]
(println shared)))
#2023-03-1615:54markaddleman(btw, I’m not criticizing, I’m just trying to understand the current state of the world)#2023-03-1615:55Dustin Getzyou should move the destructure to the client and stream the leaf#2023-03-1615:55markaddlemanah, that makes sense#2023-03-1615:55markaddlemanperfect#2023-03-1615:56Dustin Getzsure we could optimize let, haven’t thought about it much#2023-03-1615:57markaddlemanI don’t have a strong opinion on it. The fact that I can move the destructure to the client gives me exactly what I was looking for#2023-03-1615:26Yab MasI could use some feedback on the following. I'm trying to setup a project that uses js-modules and woud like to abstract the mounting of the program so I can call this from the init-functions of the modules with their root-component.
I now have:
(defonce ^:private reactor nil)
(defonce ^:private current-app nil)
(defn ^:private mount-current-app []
(set! reactor (current-app
#(js/console.log "Reactor success:" %)
#(js/console.error "Reactor failure:" %))))
(defn initialize-app [app]
(set! current-app app)
(mount-current-app))
(defn ^:dev/after-load start! []
(assert (nil? reactor) "reactor already running")
(mount-current-app))
(defn ^:dev/before-load stop! []
(when reactor (reactor))
(set! reactor nil))
and then for each module a file with something like:
(defn ^:export init! []
(current-app/initialize-app
(e/boot
(binding [dom/node js/document.body]
(hello-world/HelloWorld.)))))
They're all cljs files.
This seems to work fine at first. But I noticed hot-reload is now broken. As in, it still reloads, I see the stop! and start! logs but it mounts the old-program. I'm also wondering if it's possible to abstract this last bit of duplication as well, the call to e/boot, so I can just pass the root-component. I tried, but the results of e/defn seem to be nil when passed around like that in normal cljs, not sure if that suprises me or not, still trying to get my head around the basics. Any hints appreciated 🙏#2023-03-1615:35Dustin Getztry lifting out the e/boot to a def#2023-03-1615:52Yab MasDoesn't change anything... So still works, but doesn't properly update on hot-reload#2023-03-1615:52Yab Mas(def root
(e/boot
(binding [dom/node js/document.body]
(hello-world/HelloWorld.))))
(defn ^:export init! []
(current-app/initialize-app root))#2023-03-1615:53Yab MasThought you might have meant e/def but that just causes an error#2023-03-1620:11xificurCdo you have a ^:dev/always on the user ns https://github.com/hyperfiddle/electric-starter-app/blob/a8463f499f05397d7a567491bca1e12f512ee05d/src/user.cljs#2023-03-1620:12xificurCthe fact that it mounts the old program suggest your changes might not be getting picked up for some reason#2023-03-1707:42Yab MasYes, I've got ^:dev/always. Actually on both the current-app ns and the init ns as I wasn't sure what was necessary . I just now tried removing it from either one of them, but it all gives the same result. I also see in my browser logs that they all got reloaded. That is: hello-world, current-app and init. One thing that stands out is that the order is: hello-world > current-app > init, while I would expect hello-world > init > current-app. Im not sure if the printing order is guaranteed to be the reloading order, but this seems to be consistent on all reloads. If this had anything to do with the observed behaviour I would actually expect the old update to show up when I trigger another reload with a new change, but it doesnt... it just always loads the first version. Complete refresh obviously updates to the latest version.#2023-03-1708:37Yab MasI'm also still very interested in the second part of my question. Not so much because I necessarily need to pass electric-components around in cljs file, Im fine with having it refactored to this point, Im just trying to understand what is/isnt possible and why. In my cljs-repl eval of HelloWorld gives nil, so thats inline with the errors I get when Im trying to pass it around as a value. Macroexpanding also result in nil. Eval of (e/boot ...) gives me an object, so it makes sense Im able to pas that around in cljs-land. Macroexpanding it gives... well, something that doesnt really help my understanding 🙂 I guess its cljs side of the reactive-runtime. Not sure how to integrate my findings into actual understanding.#2023-03-1708:47xificurCwhat do you mean by "eval HelloWorld"?#2023-03-1708:48xificurCcould you provide a repo with repro steps to further investigate your issues?#2023-03-1708:51Yab Massure, I'll clean up a bit and sent you the repo with some instructions, thanks!#2023-03-1708:53Yab MasI just meant evaling the var and checking the result in the repl. Another way to put it (def x hello-world/HelloWorld) whats the value of x in a cljs file, nil apparently#2023-03-1709:03xificurCok, understand. The reason for that is e/def doesn't actually do anything 🙂 It stores the code as metadata on the var. The electric analyzer will look at that metadata and compile your program#2023-03-1709:23Yab MasI see. When I eval (meta #'hello-world/HelloWorld) I find your meta-data in the result. I'll have to think about the implication for a bit, but this definitely gives me a better understanding, thanks#2023-03-1710:37Yab Mashttps://github.com/avisi-apps/tech-testing-ground/tree/master/prototypes/electric. Start repl, function to start shadow-watch is in dev > user. Run mount/start in src > server > server. Locahost:3000 now shows hello world. Changing something in src > components > hello-world and saving triggers reload but doesnt update whats shown in the browser.#2023-03-1715:01Geoffrey GaillardThank you for the repo. It helps a lot. We are swamped today, sorry, we will look into it first thing on Monday morning 🤞#2023-03-2009:59Geoffrey GaillardI understand you want multiple js modules with one electric program per module. Shadow's :init-fn runs once when the module is loaded, but not during hot code reload. As a result, the init! functions in main-page and current-app won't be called on hot code reload. You can either:
• invert the dependency between current-app and main-page or item-view
• add ^:dev/after-load to one of your init! functions.#2023-03-2012:58Yab MasOk, makes sense. I think it's actually connected to the second part of my initial post. I tried to copy this structure from the way we do it in our fulcro apps, but there I'm able to pass the component as an arg to inititialize-app and changes to it get picked-up on hot-reload. As I'm unable to pass electric-components around like that and have to boot them first I got this situation.
After learning last friday that for electric-components it's about the var/meta-data and not the value, I tried sending it as a symbol with the intent of finding the corresponding var from wihtin the other ns but this doesnt seem possible in cljs (I might be wrong about that). I guess it should be possible to extract the relevant meta-data, pass that as an arg and than reconstruct a var in the other ns which can be passed to e/boot, but it feels a bit contrived so havn't tried (and I now actually think it wouldnt solve the reload problem either).
As for the reload-problem itself. The first option probably works but breaks the code structure, so I went with the second and it works fine, thanks for looking into it.#2023-03-1617:36JHi! Is this code valid:
(dom/on "click" (e/fn [e]
(.preventDefault e)
(try
(e/server
(e/offload
#(let [foo "foo"]
(prn "BAZ")
(Thread/sleep 10000)
(prn "FOO")
foo)))
(catch Pending _
(swap! !state assoc :status :submited)
(e/on-unmount #(swap! !state assoc :status :idle))))
I never see the FOO print. Like if the Thread/sleep was ignored. I don’t understand why.#2023-03-1617:44Dustin Getzhmmm I dont know, and can reproduce#2023-03-1617:50Dustin Getzstill thinking#2023-03-1617:51Dustin GetzIn case you're stuck, just don't silence the Pending exception inside the callback and it will work#2023-03-1617:52Dustin Getzdom/on is listening for Pending to know when the callback has successfully run on the server#2023-03-1617:56JWhat do you mean by don't silence the Pending exception?#2023-03-1617:56Dustin Getzyou're catching it (i.e. not rethrowing it)#2023-03-1618:36JThanks @U09K620SG#2023-03-1619:15Dustin GetzYou can use this pattern to hook the loading state#2023-03-1619:16Dustin Getz#2023-03-1619:16Dustin Getz(case (e/server ...) X) is the interesting part, the case here only runs the body X when the pending state is resolved#2023-03-1619:17Dustin GetzDoes that make sense?#2023-03-1619:17Dustin GetzWe are still thinking about this usage of case and how to do this better#2023-03-1619:59xificurCin this case you could also just use e/on-unmount in the body#2023-03-1710:02Jhummm it’s an unusal case ^^. The try/catch is more explicit.#2023-03-1710:11xificurCin this case the try/catch is not needed at all. When the event handler finishes it is unmounted, so you can use on-unmount#2023-03-1710:49JOoh ok I see! #2023-03-1617:56Canhello! How can I move that button to left button? I checked some CSS tutorials but everything didn't work that's what I tried until now.#2023-03-1618:37JDo you use flexbox css api? (https://css-tricks.com/snippets/css/a-guide-to-flexbox/)#2023-03-1619:55xificurCcan you share your code?#2023-03-1620:54Canof course#2023-03-1620:55Can(e/defn SliderApp []
(e/client
(let [!state (atom {:in "" :v 0 :v-state 0 :placeholder "Write a number please..."})]
(let [in (get (e/watch !state) :in) v (get (e/watch !state) :v)]
(dom/div (dom/props {:style {:display :grid
:width "40em"
:grid-gap "0.5em"
:align-items :center}})
(dom/h1 (dom/text "Slider Example")
(dom/props {:style {:grid-row 1 :align-items :center
}}))
(ui4/range v (e/fn [newv] ((swap! !state assoc :v newv)))
(dom/props {:min 0, :max 100, :style {:grid-row 2}}))
(ui4/input in (e/fn [v] (swap! !state assoc :in v))
(dom/props {:placeholder (get (e/watch !state) :placeholder)
:style {:background-color (get (e/watch !state) :bg-color2)
:width "47em" :height "1em" :align-items :center :grid-row 3}})
(dom/on "keydown" (e/fn [enter]
(when (= "Enter" (.-key enter))
(when-some [givenValue (contrib.str/empty->nil (-> enter .-target .-value))]
(swap! !state assoc :v givenValue)
(set! (.-value dom/node)
)))))
(dom/on "keyup" (e/fn [keyup]
(when-some [givenValue (contrib.str/empty->nil (-> keyup .-target .-value))]
(swap! !state assoc :v-state givenValue)
)))
)
(dom/button (dom/on "click" (e/fn [click] (swap! !state assoc :v (get (e/watch !state) :v-state))))
(dom/text "Insert Num!!!")
(dom/props {:style {:grid-row 4 :width "15em" :height "2em"
:grid-gap "10em" :align-items :auto
}})
)
(dom/button (dom/on "click" (e/fn [click] (swap! !state assoc :v "")
(swap! !state assoc :in "")))
(dom/text "Reset!!!")
(dom/props {:style {:grid-row 4 :width "15em" :height "2em"
:grid-gap "1em" :align-items :auto
}})
)
(dom/h1 (dom/p (dom/text "result is: " v))
(dom/props {:style {:grid-row 5 :align-items :center
}}))
)
)
)
)
)
#2023-03-1708:50xificurCyour issue probably stems from bad CSS / grid usage#2023-03-1709:10xificurCfrom some quick fiddling it seems you putting 2 items on a row creates 2 grid columns. I don't think grids give you much here, but the minimal change to put the buttons together is to wrap them in a div#2023-03-1709:14xificurChttps://jsfiddle.net/42gm5pf9/1/#2023-03-1618:46denikGetting this often when typing fast using a keydown listener (dom/on "keydown" ….)#2023-03-1619:01Dustin Getzthis is harmless, we're dealing with it#2023-03-1707:13nivekuilbest way to bring your own transit handlers?#2023-03-1707:48Yab MasJust answering because it might be a few hours before the team wakes up, they might have other ideas, but it's fairly easy to add the electric-ws-handler to your own ring-jetty-server and then you can do whatever you want.#2023-03-1708:03Yab MasJust make sure to read https://clojurians.slack.com/archives/C7Q9GSHFV/p1678902874125889?thread_ts=1678901838.657109&cid=C7Q9GSHFV if you go with jetty 10, else you'll run into the same issue i did the other day, I believe the examples for 9 are up-to-date#2023-03-1709:17Geoffrey GaillardI was planning to give a look at custom transit handlers today. Good timing 🙂 I’ll keep you posted#2023-03-1709:19Geoffrey GaillardYab I think you are talking about ring handlers, whereas Kevin wants to customize how values are serialized on the wire. Or am I confused?#2023-03-1709:25Yab MasNah your right, I misunderstood#2023-03-1717:33Geoffrey GaillardWe have made progress. We are figuring what’s the right API and we will release customizable transit handlers in the near future.#2023-03-2011:36Geoffrey GaillardPushed custom transit handler support to https://github.com/hyperfiddle/electric/commit/047af21723fb315e6aef2791a302b11d30148804.
See https://github.com/hyperfiddle/electric/blob/047af21723fb315e6aef2791a302b11d30148804/src-docs/wip/demo_custom_types.cljc.
Note the comment in the example code. Keep us posted if the current approach is not compatible with your use case.#2023-03-1711:50tatutI’m seeing weird errors that the simplest possible code gives unresolved symbols when used in e/server
Encountered error when macroexpanding hyperfiddle.electric/boot.
Unable to resolve symbol: let
things like let and println and I don’t understand why#2023-03-1712:02Geoffrey GaillardCould you share some code? Seeing the file would help#2023-03-1712:07tatutfor example here I have a component for selecting or creating an entry
(e/defn SelectOrCreate [{:keys [table where display-key id-key empty-option create-option template]
:or {display-key :ds/name
id-key :ds/id}}]
(e/server
(let [tbl 0 #_(new latest-table-change> table)]
(e/client
(let [options (fetch. table #{id-key display-key} where tbl)
!creating? (atom false)
creating? (e/watch !creating?)
create-val (str (gensym "new"))]
(<% :div.relative
(<% :select.select {:on-change (e/fn [e]
(let [v (-> e .-target .-value)]
(reset! !creating? (= v create-val))))}
(when empty-option
(<% :option {:disabled true :selected true} (dom/text empty-option)))
(e/for-by id-key [item options]
(<% :option {:value (get item id-key)}
(dom/text (get item display-key))))
(when create-option
(<% :option {:value create-val} (dom/text create-option))))
(when creating?
(<% :div.absolute.bottom-0.left-0
(InputGroup. {:label create-option
:name "Luo"
:placeholder create-option
:on-enter (e/fn [val]
(reset! !creating? false)
(new insert! table (template val)))})))))))))
even with that simple let binding to 0, it fails… if I take out the let and e/server from the top, it works#2023-03-1712:47xificurCI copied this, stubbed out functions you didn't provide, and it compiled for me. I'd next restart everything or start removing code to see at which point does it fail#2023-03-1712:49xificurCalso, aren't fetch and insert! server functions?#2023-03-1712:50tatutThey are e/defn that have e/server blocks#2023-03-1714:13Geoffrey GaillardIs the code you shared in a .cljc file?#2023-03-1714:58tatutYes#2023-03-1715:12Dustin Getzproblem may be at entrypoint#2023-03-1715:22Dustin Getzprobably next step is to send us your whole repo and see if we can reproduce#2023-03-1715:23Dustin Getzwhat is latest-table-change>#2023-03-1715:23tatutIt is commented out, so shouldnt affect #2023-03-1715:28Dustin Getzoh i see#2023-03-1715:28Dustin GetzCan you minimize it down to an entrypoint + a few LOC?#2023-03-1715:29Dustin GetzI wonder if a userland macro could cause this#2023-03-1715:36tatutI’ll try to get a minimal repro#2023-03-1805:03tatutmust be a userlang macro, because even this fails
(e/defn SelectOrCreate [{:keys [table where display-key id-key empty-option create-option template]
:or {display-key :ds/name
id-key :ds/id}}]
(e/server
(let [tbl 0]
(e/client
(<% :div.foo "foo" (dom/text tbl))))))
I’m referring to code that uses my specql library, that does very heavy macro stuff… something in my setup confuses it#2023-03-1805:11tatutthe backend side touches lots of code that has macros (like the clojure.spec.alpha)#2023-03-1807:26xificurCDoes it fail without <℅?#2023-03-1807:34tatutthe <% works in other places that use server and client#2023-03-1921:34Dustin Getzpush your repo and I will take a look#2023-03-2114:16Dustin Getz@U11SJ6Q0K one way to work around any potential issue in Electric is to wrap all heavy macro calls in an ordinary clojure function, instead of calling them from Electric regions#2023-03-2114:18tatutthe macro namespace in question does some very heavy lifting, and expands to a bunch of clojure.spec definition calls at ns load time, so not really an option to do that from a function#2023-03-2114:18tatutI’ll come back to this later, for now I made this app in another way, so not critical#2023-03-2114:22Dustin Getzwere you calling the heavy macros from Electric? It seems like you just have a namespace somewhere (not related to electric) with macros in it?#2023-03-2114:22tatutI wasn’t calling that, the compiler just seemed to get confused whenever some code referred to the namespace that had the macro calls in it#2023-03-2114:23Dustin Getzi dont really undertstand what that means#2023-03-2114:24tatutso in my .cljc file containing electric components I have a :refer [some.ns] at the ns declaration#2023-03-2114:24tatutand if some.ns has the macro calls, it seems to get confused#2023-03-2114:24Dustin Getzyou mean :require ?#2023-03-2114:25tatutI understand this is not a very good bug report, but I’ll come back to it later when I have some time#2023-03-1814:05Dustin GetzWe're doing a hack week next week, any ideas as to what we should build? I list some ideas here: https://twitter.com/dustingetz/status/1637081011761684480#2023-03-1816:26Andreas EdvardssonCooperative spreadsheet app, possibly using
https://github.com/pixelated-noise/hike
https://www.pixelated-noise.com/blog/2022/03/01/hike/index.html#2023-03-1817:23Lidor CohenMega like 👆#2023-03-1820:48Dustin Getz@UEJ28A9PH help me imagine what this spreadsheet would be used for, is this just a simple google sheets thing where you can add and remove rows and assign values? What is the business use case that you have in mind#2023-03-1823:32Dag Norberg- Electric DataScript
- Electric DAG viewer
Really interested in these two!!#2023-03-1907:53Andreas EdvardssonBacisally a google sheets clone with som simple formulas and stuff. I didnt have a business case in mind, just thought it would be cool. Especially if it was still performant with somewhat large data sets :) However, the other ideas are indeed more useful 👍 #2023-03-1908:06Lidor CohenHow about instead of using some kind of mock excel language (=IF(...)) we'd "sink" back to clojure? I.e formulas would be clojure exp which can reference other cells in the spreadsheet?
Besides the obvious power proof for electric, a spreadsheet that can tap into clojure, IMHO, would be priceless...#2023-03-1909:44Adrian SmithI'd like to see business applications with forms validations, form sessions, multipart forms, dialogs, file upload, consideration for conflicting updates, form history - everything you'd want to do a generic web form, defined declaratively for easy implementation#2023-03-1917:29teodorluI'd like to see an extensible, lightweight database UI. Connect to datasource (next.jdbc?), browse around.
Could be interesting as a sidecar when developing. Seeing the database state live, possibly extending the viewers. Easy batch editing.#2023-03-1818:47bedersHello there, getting my feet wet with electric and I’m already trying something impossible 😉
I’d like to use the highlight.js library and looking for a way to mutate a DOM element after it has been mounted.
The MDN answer is: use MutationObservers, but for those I also need a reference to the DOM element.
Any idea on how to do this?#2023-03-1818:57beders(I’ve spotted the on! and on fns/macros but there’s no suitable DOM event anymore to attach to)#2023-03-1819:44xificurCCan you elaborate a bit on the use case? Their web page shows simple examples that should work in electric as is#2023-03-1820:45Dustin Getzdom/node is in dynamic scope, it is the node at current point#2023-03-1820:46Dustin Getzmount and unmount looks like this: https://github.com/hyperfiddle/electric/blob/cd748d059c0035cbbb8140f4c829b200fc374d36/src-docs/user/demo_chat_extended.cljc#L52-L53 ; note that mount is the same as first render#2023-03-1821:38bedersthank you so much for the pointers!#2023-03-1903:38bedersgetting closer and what I really need is an effect that runs when a node updates (so I can ask highlight.js to highlight the current node again)
So stuff like this works:
(dom/code (dom/props {:class "language-clojure"})
(dom/text "{:a :b}")
(highlight/highlightElement dom/node))
but this won’t, since it doesn’t re-run highlightElement.
(dom/code (dom/props {:class "language-clojure"})
(dom/text (e/server (pr-str project)))
(highlight/highlightElement dom/node))
Which should be obvious to me if I would know electric a bit better 😉#2023-03-1903:39bedersI haven’t found an example yet that shows how to integrate with one of those good old-fashioned JS libraries that want a dom element to mess with (and don’t observe any DOM changes themselves)#2023-03-1904:04bedersswitched to a WebComponent and that works. No need to investigate further.#2023-03-1914:17Dustin GetzI dont understand the webcomponent fix, can i see the code?#2023-03-1914:20Dustin Getz> what I really need is an effect that runs when a node updates (so I can ask highlight.js to highlight the current node again)
Here is a solution, it requires you to think in terms of the DAG / reactivity graph
(defn highlight! [node _]
(highlight/highlightElement dom/node))
(dom/code (dom/props {:class "language-clojure"})
(dom/text (e/server (pr-str project)))
(highlight! dom/node project)) ; establish reactive dependency on project#2023-03-1921:39bedersOh. I see. I have to reference project#2023-03-1921:41bedersI’ve since switched to a web component that takes the data from project directly and that makes it reactive
#2023-03-1921:42Dustin GetzI'd like to see it [webcomponent] if it is easy to copy/paste#2023-03-2004:46bedersI’m using this one: https://github.com/alenaksu/json-viewer
Here’s my example component.
Surprisingly it can deal with CLJS objects.
(e/defn JSONView []
(e/server
(binding [project (e/watch d/project)]
(e/client
(dom/div
(dom/element "json-viewer"
(dom/props {:data (e/server project)}))
)))))
(calling :data (cljs->js (e/server project)) fails with an error message for reasons unknown)#2023-03-2004:46bedersif project changes, it re-renders. Very fast.#2023-03-2004:49bedersjson-viewer doesn’t quite fit the bill as keys are unsorted, so I might build my own viewer for tree-like data-structures anyways. Sigh. Too much effort to fork json-viewer, change one line, and get it into the project 😉#2023-03-2111:51Dustin Getz@U4YGF4NGM started an electric tree viewer, it's in https://github.com/hyperfiddle/electric/blob/master/scratch/photon_tree.cljc#2023-03-1821:38nakkayaHi, I use the following for history (slightly modified from electric examples.) [1] This used to work no issues since switching to master I started getting the following error, back/forward navigation still works but I lost the ability to reload the app from a specific view. i.e /app#/some-page
[1] https://gist.github.com/nakkaya/fb5acbebf8ff7a4f468fa92b6a0b7342#2023-03-1923:47Dustin Getzi have no idea, will look into it tomorrow, if you can link me to the right place in your repo that would help #2023-03-2000:55nakkaya👍 Thanks for looking into it. Here is history.cljc [1] and core.cljc[2].
[1] https://github.com/nakkaya/dentmetria/blob/main/src/app/history.cljc
[2] https://github.com/nakkaya/dentmetria/blob/main/src/app/core.cljc#2023-03-2009:01xificurCI get page not found on your links#2023-03-1821:39Dustin Getzfrom @sr (who did Electric Painter) - one day#2023-03-1823:50Dag Norberghttps://electric-clojure-rpg-production.up.railway.app/#2023-03-1900:50Dustin GetzThe demo (and other game demos) has input latency, I spoke with Dag about it. We're going to fix it. The issue is that Electric's operators are designed for database apps, Electric has optimistic form controls but games don't match the form pattern#2023-03-1902:39nivekuilmore info on how you plan on doing that? sounds like a major undertaking#2023-03-1902:41nivekuilform inputs are easy because the user is the source of truth#2023-03-1913:33Dustin Getzwe don't intend to abstract over conflict resolution, which is the hard part right? userland can arrange a transaction or CRDT strategy#2023-03-1913:35Dustin Getzabstractly, a form is a sequence of keyboard events, and a mouse gesture is a sequence of mouse events, and a game input is a sequence of keyboard events, I think the naive interactions can be fast like our form elements are#2023-03-1913:37nivekuilIs "clicked on a thing" a mouse event? server and client may disagree on whether that happened#2023-03-1913:37Dustin Getzthat's a conflict, we don't intend to abstract over that#2023-03-1913:39Dustin Getz+ we need a user story, "clicked on a thing" is a fact, and the client has authority over that fact#2023-03-1914:13Dustin GetzWe also just discovered two additional issues
1. Railway has 200ms ping across the atlantic ocean whereas http://fly.io has 30ms for the same
2. in the painter app, DOM performance slows down linearly with number of strokes, this is a userland issue that i will help fix
and of course (3) the optimistic state update issue remains (in games not forms) even with low ping, we take that seriously#2023-03-2816:09Dustin Getz@sr we fixed the glitchy input, it was a mistake in the userland code in the RPG app, sent you a PR#2023-03-1823:50Dag Norberghttps://electric-clojure-rpg-production.up.railway.app/#2023-03-1919:34Dustin Getzanyone want to port this? https://github.com/enricoros/nextjs-chatgpt-app#2023-03-1919:58adamfreytoday working on some more recent electric git shas I've been seeing
hyperfiddle.electric-jetty-adapter: Websocket handler failure
clojure.lang.ExceptionInfo: Unable to resolve symbol: datascript.core/transact!
which is a bit confusing, since that var should definitely exist. Other times it will be a different error complaining that the first var in my ns doesn't exist:
(defonce !conn #?(:clj (d/create-conn {}) :cljs nil)) ; database on server
but yet,
Unable to resolve symbol: <...>/!conn
this error occurs when I load localhost:8080 in my browser for the first time after starting the compiler build
The shas I've see it on are: eb959daf49bc0b26feee613453dce27ae3375314 ce7da1b0859da24758dbfb81cf0ae7a299cc7b2a b3b8d2e823f71e5416d0a93a94c405d5e2a8490e
has anyone else seen this?#2023-03-1920:06Dustin Getzcould you be missing an e/client or e/server?#2023-03-1920:13adamfreyI'll verify by commenting out most of my app.
I do think what I have right is mostly just a couple tweaks to the starter app todo list.#2023-03-1920:14Dustin Getzpush it and i will clone and take a look#2023-03-1920:21adamfreyhttps://github.com/AdamFrey/electric-starter-missing-symbol
• clj -A:dev -X user/main
• localhost:8080
if you don't see the error on the first time, restart the server and then load the page again#2023-03-1920:44Dustin GetzI repped, investigating#2023-03-1921:19Dustin GetzI solved it, you are going to be unhappy#2023-03-1921:20Dustin Getzuser.cljs, change (ns user ^:dev/always to (ns ^:dev/always user#2023-03-1921:20Dustin Getzthis one is on us, we had it backwards in the starter app for a bit#2023-03-1921:23Dustin Getzyou even made a fine effort to upgrade your fork of the starter app, including when you bumped to master, somehow realizing to add ring-basic-authentication/ring-basic-authentication {:mvn/version "1.1.1"} to your app which is missing from the starter app currently#2023-03-1921:24Dustin Getzhere is the https://github.com/hyperfiddle/electric-starter-app/commit/a8810accfdd96f82eefc2d976645538223a88de9#diff-06a7049242ecf7dac4e22b30db9faa22ebae4e22e72d1bfbb916e03e3075e5c1 you would have needed to see, very unhelpfully labeled "upgrade electric"#2023-03-1923:25adamfreyMaking mistakes with clojure metadata placement: an intractable problem?
I always have a hard time remembering where it should go.
Thanks for looking into this for me. I’ll apply the fix when I get back home.#2023-03-2013:47Dustin Getzhttps://electric-demo.fly.dev/ posting our examples here as a load test, let me know what the experience is like#2023-03-2013:48Dustin GetzNote - Todomvc-composed has a crash, we're aware of it#2023-03-2013:53tatutpretty good, but I can notice a little latency (for example in todomvc and file explorer scrolling)#2023-03-2013:53tatutI’m in Finland if that matters, what region is this in?#2023-03-2014:14Geoffrey GaillardThe closest deployed node is in Paris#2023-03-2014:19Dustin Getz@U2DART3HA are there multiple instances of this? i thought there was just 1 in EWR#2023-03-2014:20Dustin Getznow i see 2, was that automatic?#2023-03-2014:21Dustin GetzI am not seeing latency, the node_modules scrolling is the fastest non-local virtual scroll i've ever seen#2023-03-2014:22Geoffrey GaillardI see one instance in ewr and one in cdg. The monitoring page is not 100% reactive. Try to refresh.#2023-03-2015:34chromalchemyLooks good to me. Not totally seamless, because I can see a quick flicker of the repainted rows filling in asynchronously. But feels very immediate and 100% usable!#2023-03-2015:47chromalchemyAbove comment was for https://electric-demo.fly.dev/(user.demo-explorer!DirectoryExplorer), not virtual scroll example, which does pop faster… I don’t see cascade of loading.#2023-03-2016:54Dustin Getzhow quick? My performance is the same as this video: https://gist.github.com/dustingetz/dd67a35d818e3a1bf6733147cf5cdea7#2023-03-2017:20chromalchemyI’m getting a bit more jank?
Im in Florida on 500mb connection.#2023-03-2017:22Dustin Getzweird, i haven't seen that flickering before#2023-03-2120:31denikseeing this on my end too. it’s only present on the folders/nested views for me#2023-03-2104:14bedersI’m managing to produce compilation errors when editing a CLJS file that I’m requiring via #?(:cljs)
from another ns (which contains various e/defn).
Unable to resolve symbol: app.splunk-view/TimelineView
{:file "user.cljs", :line 10, :column 7, :end-line 10, :end-column 38, :in [(app.splunk-view/TimelineView.)]}
ExceptionInfo: Unable to resolve symbol: app.splunk-view/TimelineView
The curious thing is, that if I just $ touch splunk_view.cljc
the compilation error goes away and things are back to normal.
What’s the best way to troubleshoot this? What details can I provide?#2023-03-2104:16bedersElectric namespace:
(ns app.splunk-view
(:require #?(:clj [app.data :as d]) ; database on server
#?(:clj [cheshire.core :as json])
#?(:cljs [app.ui :as ui])
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
))
app.ui is a cljs file which does evil and verboten things (uses hiccups to create HTML that I then attach to a dom/node#2023-03-2105:15bederswhen I change app.ui and shadow-cljs does its thing, that’s when the compilation error occurs.#2023-03-2105:35bedersnever mind. A missing require was the root cause#2023-03-2115:57bedersI’m still quite a noob wrt missionary and how it is used in Electric, so excuse me if these are dumb questions. I’m coming from a reagent background and have avoided monads due to their infectious disease nature.
The DSL electric provides makes monads almost disappear which is great!
Here’s a use cases for which I’m not sure how to proceed:
If I’d like to use third party JS libraries and render some of my content directly (similar to what the React Wrapper does).
How can I bridge from events coming from those 3rd party libraries into electric?
i.e. running some (e/server …) code#2023-03-2116:14Dustin GetzI think the https://github.com/hyperfiddle/electric/blob/7ef1612c9659d1e88f2f41223f5c7c045a5eb399/src/contrib/electric_goog_history.cljc#L31-L35 shows what you want#2023-03-2116:15bederscool, thank you. Haven’t looked at all the examples yet.#2023-03-2116:15Dustin Getzyou wouldn't have found this one#2023-03-2116:16Dustin Getzm/observe is the key primitive, it lets you subscribe to a foreign resource, with a callback ! to bridge the foreign events to the missionary flow, and also has a cleanup handler#2023-03-2116:16bedersare you aware of anyone retro-fitting a variation of hiccup to eletric?
Currently playing around with a macro that takes hiccup and emits dom snippets#2023-03-2116:17xificurCIf you can get a handle of the DOM node you wish to work with you can bind dom/node to it and call electric code on it#2023-03-2116:18Dustin Getz@U628K7XGQ there have been several attempts at a hiccup macro but they all handle only the happy path, no edge cases. Can you ask a more specific question?#2023-03-2116:19xificurChttps://github.com/hyperfiddle/electric/issues/32#issuecomment-1476316901#2023-03-2116:19bedersI’m just longing for the better ergonomics of hiccup vectors (in the reagent flavor - without adopting the reactive nature of reagent#2023-03-2116:20bedersthanks for the link, Peter#2023-03-2116:20Dustin GetzThere are some people using and happy with unsupported hiccup wrappers but we do not support it, I also don't recommend you do it#2023-03-2116:22xificurCto support your third party library it'd be best to provide an example so we can help you write the simplest / most idiomatic glue code#2023-03-2116:23bedersthis is to cover use cases where I have pretty big objects that need rendering but don’t have any reactive parts in it (i.e. it is ok to re-transmit the big object if it happens to change. I don’t need reactivity all the way to the leaves).#2023-03-2116:24bedersWriting the render code with (dom/… (dom/props …) (dom/text …)) vs the hiccup equivalent makes quite a difference in readbility#2023-03-2116:25bedersI have hooked up the original hiccups library which emits an HTML string and used .innerHTML to render it. It’s plenty fast but there’s no event handling of course, hence the question#2023-03-2116:32Dustin Getzyou're right that the low level syntax is heavy and not great, we will provide a great syntax as soon as we can#2023-03-2116:47bedersthanks for the quick help! I’ll manage with a few helper fns for now.
And I must say: very impressive stuff you guys built. Kudos#2023-03-2216:40Michael WIs there a way to get electric to tell me which line an unserializable transfer has occurred? I keep missing places that I need to add (e/server) and since it gives me the datom from datascript rather than a line where it occurred it's a bit hard to track down where in my code the issue is.
INFO : Unserializable reference transfer: datascript.db.Datom #datascript/Datom [8 :player/session-id "MzBVM0RORlhRK09zSDQvcmZQMDFIdz09" 536870921 true]
I'm using the same datoms all over my code so it's not exactly clear where the problem is happening.#2023-03-2217:18xificurCThis has been on our backlog for a while. For now I typically bisect-comment the code in question#2023-03-2219:01denikhave been building a clojure spredsheet with electric#2023-03-2219:02denik#2023-03-2219:02denikcells updates trigger reactively when referenced cells update. all execution happens in parallel on separate threads#2023-03-2221:32teodorluwow, nice!
how did you like using electric for building it? did you face any problems?#2023-03-2317:25denikloved it! no problems. I had tried building this in reagent in the past and it quickly turned into a serialization/performance nightmare#2023-03-2317:25denikwith electric those problems are non-existent#2023-03-2317:26denikthe vibe#2023-03-2317:27denik#2023-03-2410:30teodorluNice!
Spreadsheets and electric seem like a great match to me too, but I've never tried myself!#2023-03-2220:40Carsten BehringHow can I relocate an electric app to an other context root, not "/" ?
Is there something in the "build" to do so ?#2023-03-2220:55Dustin GetzElectric itself doesn't know about the page's path. There are two places in userland application code that relate to the path:
• index.html or other userland page entrypoint which boots the Electric client, e.g. https://github.com/hyperfiddle/electric-starter-app/blob/main/src/user.cljs#L16-L18 - you can call this from anywhere, Electric doesn't care, it will connect by websocket protocol and doesn't interact with any ordinary http endpoints (IIUC, the team will correct me if I made any mistake)
• Application router code done from inside the Electric application. The starter app doesn't even include a router and never looks at the path (browse around the starter app src https://github.com/hyperfiddle/electric-starter-app/tree/main/src). The electric demo app does include a router https://github.com/hyperfiddle/electric/blob/7ef1612c9659d1e88f2f41223f5c7c045a5eb399/src-docs/user_main.cljc#L88-L97, so you'd need to slice off any context at this point.#2023-03-2220:56Dustin GetzDoes this answer your question?#2023-03-2220:59Dustin GetzNote, I do not recommend you use the hyperfiddle.history router that the demo app uses, it is experimental and more complex to configure than we like. I'd recommend a simple https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc, you can modify that to deal with chopping off any parent routes#2023-03-2221:22Carsten BehringI was wondering where this
https://github.com/hyperfiddle/electric-starter-app/blob/0b4ebdfe6399db8aceaef6539f03c7ae7ac444ba/resources/public/index.html#L11
gets changed into "/js/main.57678446D54FAE742D3AB51E6C9B1E63.js"
When I put the starter app into Kubernetes and and an ingress , I would need to download the js from something like "/myapp/js/,,,,,"#2023-03-2221:26Carsten BehringBasicaly I would lkie that my electrip app is available at:
👉 App server available at
instead of
👉 App server available at
#2023-03-2221:33Dustin GetzIs this for at work?#2023-03-2221:34Dustin GetzIn the starter app, index.html is served by a ring middleware https://github.com/hyperfiddle/electric-starter-app/blob/0b4ebdfe6399db8aceaef6539f03c7ae7ac444ba/src/electric_server_java8_jetty9.clj#L101. The dev/ops team will need to modify the jetty server starter to suit their customizations. The starter app is NOT a production server, it is just an example, real apps need more secure auth for example#2023-03-2300:37nakkayaIf you serve index.html via /myapp route. electric will be available via /myapp and you use / for something else. You can use your own ring handlers my app lives in /app.#2023-03-2311:19Dustin Getz> Q: where is <script type="text/javascript" src="$:hyperfiddle.client.module/main$"></script> rendered?
see template in jetty-server https://github.com/hyperfiddle/electric-starter-app/blob/0b4ebdfe6399db8aceaef6539f03c7ae7ac444ba/src/electric_server_java8_jetty9.clj#L60-L62#2023-03-2322:06Carsten BehringI don't get it working. There are too many places in the starter-app to change, and probably i miss some.
Does somebody have an example app (with code) which is not on "/" but on "/myapp".#2023-03-2402:00nakkaya@U7CAHM72M This is the striped down version of my core.clj electric lives under /app and a generic landing page under / [1]
[1] https://gist.github.com/nakkaya/7b65ddc161aad9beb7207c8b520c47f3#2023-03-2311:13Dustin GetzHere's another preview (older) of @denik’s spreadsheet#2023-03-2413:43mattiasI'm encountering two issues in https://github.com/subspace-ai/subspace:
1. There is a dom/div https://github.com/subspace-ai/subspace/blob/d15601d3c04c4a07f9c01eb52b663508a8432706/src/app/nodes.cljc#L30 (which has :contenteditable true ) with a dom/text https://github.com/subspace-ai/subspace/blob/d15601d3c04c4a07f9c01eb52b663508a8432706/src/app/nodes.cljc#L33 nested within. If the initial value of dom/text is empty or whitespace, then the corresponding DOM element does not get shown in Chrome DevTools inspector (perhaps the browser cleans it out, or Electric does). Then if I edit the contents of the div in the browser to "val", and focus away from the div (which triggers an https://github.com/subspace-ai/subspace/blob/d15601d3c04c4a07f9c01eb52b663508a8432706/src/app/nodes.cljc#L59 of the corresponding DB record), then the DOM tree ends up with two text nodes with content "val". If I focus back to the div and away again, the text nodes become "val" and "valval" . I think it's because the first text node "val" is added there by Chrome's contenteditable behaviour and is ignored by Electric, and the second value "valval" is the dom/text node that Electric adds, since now that the DB gets a new non-empty value, the text node gets added and updated (it becomes "valval" because the DB update from last time used both text nodes of the div). If the initial value of the dom/text node is non-empty, everything works fine.
2. When https://github.com/subspace-ai/subspace/blob/d15601d3c04c4a07f9c01eb52b663508a8432706/src/app/nodes.cljc#L53, I'm seeing caught TypeError: Cannot read properties of null (reading 'removeChild') after the backend syncs the changes to the frontend, and the DOM elements (<li><div>) corresponding to the Node don't get cleaned out. Presumably it's expecting an entry for the now-deleted node, which is nil?#2023-03-2418:39Dustin Getzafter first read, it seems like there is some interaction with contenteditable (we had not tried this yet, you are the first to try). This would not be surprising as we have seen issues with elements being mutated in unexpected ways (i.e. adding another textnode that we didn't account for). I will take a closer look when i have time - tonight or tomorrow hopefully. I'll look at #2 as well, it could even be related#2023-03-2513:17Dustin GetzAh, #2 is fixed in master already, which hasn't been released yet on maven, you can use :git/url :git/sha#2023-03-2514:35Dustin GetzConfirmed #2 is fixed and sent PR; and the fact that the master change fixes it also confirms that there's something funky going on with how contenteditable mutates the DOM for issue #1, investigating#2023-03-2516:20Dustin GetzHere's a PR that fixes everything https://github.com/subspace-ai/subspace/pull/3#2023-03-2516:21Dustin Getzthe first PR I sent (that upgraded Electric to master) is no longer needed, as the content-editable was the root cause, so with that fixed you don't need to upgrade#2023-03-2516:25Dustin GetzAlso note we need to use an optimistic state pattern here, because we want local updates to not be delayed by round trip latency. Let's worry about that later (you will need it to deploy). We have components for this but you should wait until we release the next version of Electric in about 2 weeks which has changes here#2023-03-2605:46mattiasAmazing, thank you so much for the PR and explanations!#2023-03-2610:03grounded_sageIs there an example of using electric with Datahike? I am trying to hook it up and it's still unclear to me how to do that. I've tried e/watch on the db conn and also just binding conn directly do (e/def db) . I am sure I am missing something simple here.#2023-03-2612:15Dustin Getze/watch takes an atom-like watchable reference; DataScript connections implement those interfaces#2023-03-2612:16Dustin GetzTake a look at the xtdb starter example to see how to integrate a database that isn't watchable out of the box: https://github.com/hyperfiddle/electric-xtdb-starter/blob/69b80861b798153746be5400d83887d26e216bfe/src/app/todo_list.cljc#L69#2023-03-2617:59xificurCMaybe someone from #CB7GJAN0L will be able to help. The feature needed is to receive a new db when a transaction commits.#2023-03-2711:07grounded_sageThanks @U09K620SG I'll dig in on this. The most intuitive thing to me would be hooking into the listen function on the datahike conn though it sounds like electric wants the db-after?#2023-03-2711:15grounded_sageIf I may ask. What function does this binding serve with electric. https://github.com/hyperfiddle/electric-xtdb-starter/blob/master/src/app/todo_list.cljc#L68#2023-03-2716:26Dustin Getzwe're injecting the connection by hardcoded global ref to user/!xtdb which is a huge hack, so i wanted to contain the hack to one line#2023-03-3119:58grounded_sageI am okay with hacks. This is minimal compared to what is done to handle all the mixing of network and state management.#2023-03-3120:23grounded_sageI am mostly there with Datahike working. It's just the updates to be triggered without a refresh of the page. With the Datomic example it is a little different. Unsure where the contrib namespace is coming from? https://github.com/hyperfiddle/electric-datomic-browser/blob/main/src/app/datomic_browser.cljc#L224-L226
From what I gather you need to place (new ..) on each new value and an understanding of missionary is required to get up and running?#2023-03-3120:30Dustin Getzcontrib is inside electric jar, sorry#2023-03-3120:31Dustin GetzYes, new is like await for missionary flows#2023-04-0213:35Dave MaysRegarding the XTDB example, what does the database not being "watchable" mean?
Would the built in multiplayer depend on the database being watchable?
What has to change if you did want a watchable db?#2023-04-0213:37Dustin Getzthe database connection needs an API for subscribing to updates (e.g. a transaction log and notification). That API then needs to be adapted to Electric Clojure flows#2023-04-0213:39Dustin GetzDatascript has this out of the box
For XTDB in-memory db, this is what it looks like: https://github.com/hyperfiddle/electric-xtdb-starter/blob/69b80861b798153746be5400d83887d26e216bfe/src/app/xtdb_contrib.clj#L5
Postgres has a more complex API for this#2023-03-2612:05tobiasIs there anything we can resolve for-by as in Cursive to not get wiggly underline warnings? It's only a tiny thing but I thought I'd ask.#2023-03-2612:13Dustin GetzOnly workaround today (in Cursive) is to disable intellisense for the entire form including the body#2023-03-2612:14Dustin GetzMake your voice heard in #C0744GXCJ if you want this fixed#2023-03-2813:10sh54I have seen the demo of a reagent component hosted within electric but is the other way round possible too? I have a little app using https://github.com/ticlo/rc-dock with reagent/re-frame and I wanted to see if electric could fully control the contents of some tab.#2023-03-2817:34noonianI think this should work if you have different reactors for each tab and bind hyperfiddle.electric-dom2/node to the dom nodes for each tab. I tried this in my app and was able to have two instances of the app running in the same browser tab#2023-03-2818:41xificurCSure! You can check the starter app's https://github.com/hyperfiddle/electric-starter-app/blob/main/src/user.cljs. It binds to the document body there but you could bind the entrypoint on any DOM node, thereby embedding it. Or even running 2 electric instances. There were other users here that played around with these setups if you go through the channel's recent-ish history#2023-03-2819:37sh54Nice thank you both for the tips! I'll give it a shot#2023-03-2919:12sh54So pretty much all I needed for a proof of concept was this reagent component:
(defn electric-in-reagent
[]
#?(:cljs
(r/with-let [!reactor (atom nil)]
[:div
{:ref (fn [r]
(if r
(reset! !reactor ((e/boot
(binding [dom/node r]
;; Some electric component
(Just-the-count.)))))
(swap! !reactor (fn [r]
(r)
nil))))}])))
And building on the existing reagent interop example it seems fine to go electric -> reagent -> electric by calling (with-reagent electric-in-reagent) .#2023-03-2813:23tobiasI found some quirky behaviour with range inputs. Not sure if it's me doing something silly.
This code works as expected:
(dom/input (dom/props {:type "range"
:class "input"
:style {:width "300px"}
:min 0
:value 90
:max 100}))
It shows a slider like this:#2023-03-2813:24tobias#2023-03-2813:25tobiasBut if I increase the max value to something over 100, the slider value shows in the wrong position. For instance, let's do 900/1000 instead of 90/100:
(dom/input (dom/props {:type "range"
:class "input"
:style {:width "300px"}
:min 0
:value 900
:max 1000}))
Now, the slider looks like this:#2023-03-2813:26tobiasAlso I noticed that in both cases there is no value attribute on the input in the Chrome inspector#2023-03-2813:55Geoffrey GaillardThank you for the report. I gave it a look.
• dom/input is just a convenient syntax over a regular dom input element.
• value is not an attribute, https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input#value. It is not expected to show up in the console.
You are right that increasing :value and :max by a factor of ten should not alter what the range looks like.
I’m suspecting:
• a CSS issue (I can see custom CSS on your screenshots)
• the :value is not actually 900 - you can confirm this by selecting your <input type="range" .../> in the console and type $0.value, like this: #2023-03-2823:10tobiasI have a hunch about what's going on. I assume that Electric makes the <input> element and then attaches attributes and properties to it one by one. When the value is greater than 100, it's important that the max attribute is set before the value property, otherwise value defaults to the max value of 100.
In support of this hypothesis, I found that if I change the order of key value pairs in my dom/props map so that the value property comes last, then the slider displays correctly. Obviously this isn't a great solution, because the order of items in the map isn' guaranteed, especially if the map is larger.#2023-03-2823:11tobiasThis code (note value goes last in map):
(dom/input (dom/props {:type "range"
:class "input"
:style {:width "300px"}
:min 0
:max 1000
:value 900}))
results in slider displaying correctly:#2023-03-2823:13tobiasIt looks like this has tripped people up in other web frameworks too, e.g. this https://github.com/Megabit/Blazorise/issues/677 in Blazorise was https://github.com/Megabit/Blazorise/pull/727 by placing min and max attributes before value.#2023-03-2900:01Dustin Getznice detective work#2023-03-2900:01Dustin Getzyou can work around this by calling dom/props twice, they will be called in order of position#2023-03-2900:02Dustin Getzoh, i see the map is ordered (since the map is small)#2023-03-2900:20tobiasThanks for that - I didn't know you could put multiple dom/props calls inside a dom/input . I just tried it and it works great.
(dom/input (dom/props {:type "range"
:class "input"
:style {:width "300px"}
:min 0
:max 1000})
(dom/props {:value 900}))
#2023-03-2906:20Geoffrey GaillardIt looks like a browser bug. Are you using chrome?#2023-03-2906:40tobiasYes, using Chrome. But just now I checked in Safari and Firefox (on mac) and I see the same behaviour. Depending on whether I put the :value key before or after the :max key, the slider position is either 100 or 900.#2023-03-2906:48Geoffrey GaillardOk! Thank you very much! We'll see what we can do about it 👍#2023-03-3122:08Dustin Getztutorial - what do you think?
https://electric-examples-app.fly.dev/
#2023-03-3123:09noonianLooks great! One thing that I think should be called out explicitly is how reactivity is granular to the expression level, e.g. in this function, only the println that depends on x will be called when x changes.
(e/defn Foo [x]
(println "about to print x")
(println x) ;will print if x changes
(println "done printing x"))
I think folks coming from a React background might expect the entire body to be executed each time an argument changes because that's how React works. I think the explanation in the two clocks demo implies this behavior but it could still be beneficial to mention it explicitly to avoid faulty assumptions on the user's part#2023-03-3123:10Dustin Getzok ty i wil incorporate this#2023-04-0101:46Michael WThis looks amazing. Is there a repo for this?#2023-04-0113:40teodorluI ended up writing quite a few comments! Respond to whatever you feel like responding to.
--
General comments:
1. I’ve spent about 10 minutes staring at example 1 and 2, this is really helpful. Edit: after reading it all, it was even more helpful!
2. I appreciate that the code doesn’t show anything more than it needs to — there are no “loose ends”. Feels more “to the point” than previous Electric Clojure material I’ve read.
3. The “what’s happening”, “novel forms” and “key ideas” sections are very helpful to understand the code.
Specifics:
1. Toggle: I don’t know what “latency-stabilized” means.
2. Toggle: “Traditional “single state atom” UI pattern, except the atom is on the server” - this sentence formulates neatly why I’m excited about Electric!
3. Toggle: “Quiz: why do we call pr-str on the type number type before transferring it?”
a. I’m honestly not quite sure. Is it because values transferred over the network need to be serializable, and types aren’t?
4. SystemProperties code
a. Is (map (juxt key val)) really required? I haven’t tried in a REPL, but I’d to be able to delete that line. I could be missing something here.
5. SystemProperties: “`e/offload`: move a blocking Clojure computation to a threadpool (server only)”
a. I’m curious about why this needs to be done.
b. Is it because we want the async electric code to be able to “stuff” all the time, without waiting for any single slow thing to block? In other words, could the user interface start lagging here if we didn’t use e/offload?
c. Perhaps the form description could describe why we want to use offload, not just what it does.
d. … and I see that you actually explained this below. 🙂#2023-04-0114:39Dustin Getz@UAB2NMK25 there is, it's private currently, why do you want to see it?#2023-04-0117:37Carsten BehringMaybe make the code view "read-only".. I edited it, and nothing happened (of course...)#2023-04-0120:13Dustin GetzThank you all, I have updated the tutorial in response to the feedback#2023-04-0213:18Dave MaysTutorial is awesome, very nice to see something up and running live without needing to clone a repo or anything.#2023-04-0318:37teodorluJust read the updated tutorial, this is just getting better!
> • Unlike React.js, reactivity is granular to the expression level, not the function level.
> ◦ Each expression, e.g. (- s c), is a node in the DAG.
> ◦ Lexical scope, e.g. c, is an edge in the DAG.
> ◦ Expressions are recomputed when any argument updates.
> ◦ So, if c changes, (- s c) is recomputed using memoized s.
This piece took me a long time to understand. I think I get it now, but please correct me if I’m wrong.
1. Electric analyzes code on the expression level.
a. Each expression is async
b. Each expression is reactive
2. e/defn / e/fn create functions that Electric can analyze the body of,
3. defn work normally as it does in clojure/cljs — a normal, blocking function, opaque to Electric. Electric doesn’t mess with the clojure.core/defn macro.
4. (e/offload #(my-fun arg1 arg2)) is a way to “asyncify” normal Clojure code. We could just call Clojure code, but if it takes long to finish, it’s better to e/offload because then we don’t block Electric from updating other things while we’re waiting for results.
In other words: use e/defn to make Electric analyze individual pieces, and (my-fn arg1 arg2) to call normal functions directly, or (e/offload #(my-fun arg1 arg2)) to let the function run without blocking the electric event loop & application UI.#2023-04-0319:04Dustin GetzThat's right, I will update the tutorial with this feedback thank you#2023-04-0319:06Dustin GetzBtw we have not introduced e/offload in the tutorial yet because it causes a minor problem that we hadn't noticed, we might need to rework something. But your understanding is correct.#2023-04-0215:18AvivHey everyone, pretty new here 🙂
I’m trying to run the https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_reagent_interop.cljc demo example locally, but i’m getting this error; any idea why?#2023-04-0215:26xificurCHi! Did you run npm install?#2023-04-0215:35AvivThanks for your help!
Yes, i did,
for some reason the link for this demo is commented, https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_index.cljc
(btw i think the react & react-dom deps are missing in the package.json)#2023-04-0216:27Dustin GetzYou caught us refactoring repos, demos are being moved out of the library repo#2023-04-0216:27Dustin Getzif you wait a few days they will be here: https://github.com/hyperfiddle/electric-examples-app#2023-04-0217:10Dustin GetzI've added the Reagent demo here in a branch: https://github.com/hyperfiddle/electric-examples-app/tree/wip/more-demos, note the branch "wip/more-demos"#2023-04-0217:11Dustin Getz(I don't want to deploy the tutorials app right now while people are on it)#2023-04-0308:27AvivOh, thank you! i’ll look at the branch you sent#2023-04-0222:57Dave MaysSorry to need my hand held a bit.. Any hints on how to get the XTDB demo below to run?
https://github.com/hyperfiddle/electric-xtdb-starter
xtdb/entity in todo_list.cljc reports that xtdb.api is aliased but not loaded. Deps.edn does seem to include some XTDB already.
Terminal error reports:
Starting XTDB...
Execution error (AssertionError) at user/start-xtdb! (user.clj:11).
Assert failed: (= "true" (System/getenv "XTDB_ENABLE_BYTEUTILS_SHA1"))#2023-04-0223:04Dustin Getzthe env var needs to be set, see readme#2023-04-0223:06Dave Maysah sorry should have started there - thanks!#2023-04-0223:21Dave MaysRuns great 🙂
It's so foreign and amazing to see just 80 lines of code doing UI like this.. (not to discount what Electric is doing in the background of course).#2023-04-0312:04Dustin GetzMetrics on tutorial app are rock solid
• Constant memory usage
• linear cpu utilization
• 5x $6/mo http://fly.io instances (in 5 cities worldwide)
• blue line is ~15 concurrent sessions in Paris instance
• looks like we can serve 50 concurrent sessions on a $6/mo box with shared cpu!
• and it should scale linearly with more resources and horizontally with more instances?#2023-04-0312:12Dustin GetzNetwork I/O is the two-clocks demo, we're streaming a clock at 120hz tick rate.
• It's a stress test, real apps don't do that.
• Electric has automatic backpressure so the clock will saturate available bandwidth up to 120hz tick rate.
• It's about 10kb/s per streaming clock
• 10kB/s / 120 = 83 bytes per tick for the integer + transit overhead + electric protocol overhead.
• This is not optimized at all! We have a binary protocol as well which is both smaller and faster, which we are not even using right now#2023-04-0312:14Dustin GetzCredit to @U053XQP4S for this work on the Electric distributed runtime and of course the Missionary reactive engine. Well done Leo!#2023-04-0312:51JGood job guys 👍#2023-04-0404:19tatuthow much will the binary protocol save bandwidth?#2023-04-0410:06leonoel> how much will the binary protocol save bandwidth?
Not measured yet, but currently a large portion of the message payload is technical (i.e not user data), we expect to improve it on several aspects :
• nodes of the app structure are identified with numbers. Switching from decimal to binary encoding would consistently save 50% bandwidth
• we could get rid of a lot of delimiters by leveraging static knowledge of the message structure. It looks reasonable to save another 50% here
Another optimization we have in mind is to leverage the stateful nature of the session to cache the most frequently used values from previous messages.#2023-04-0413:57Dustin GetzWe just pushed to the tutorial app, it caused 2 mins of server outage, and clients should have gracefully recovered. Let me know if you had a tab open and had a problem#2023-04-0414:10Dustin GetzReagent interop demo is published: https://electric-examples-app.fly.dev/user.demo-reagent-interop!ReagentInterop#2023-04-0414:28Daniel JompheAlways pleasantly surprised how quickly these webpages load.
One would think Electric Clojure adds weight to the initial load time of any app to which it's integrated, but, it seems, no.#2023-04-1222:51fabraoSo, is this the only way to use react components with Electric Clojure?#2023-04-1223:28Dustin Getzthis is a quick sketch that proves high fidelity react interop is possible. how would you like to use them?#2023-04-1223:31fabraoI was wondering how to use UI Components like headless ui and I saw this as an alternative. Is there any other faster/easy way?#2023-04-1223:34Dustin Getzyou saw what as an alternative to what? #2023-04-1223:34fabraoand the other most complex like https://primereact.org#2023-04-1223:35Dustin Getzyour q is how to use a react component library with electric?#2023-04-1223:37fabraoyes, sorry about my question, not an alternative but this example as a way to use react components.#2023-04-1300:11Dustin Getzi will add it to our backlog to look at integrating one of these#2023-04-1300:12fabraothank you#2023-04-0423:10chromalchemyIs there any guidance or limitations on deploying to digital ocean vps vs http://fly.io ?#2023-04-0423:11Dustin Getzno limitations, the dockerfile should work wherever#2023-04-0423:16chromalchemyCool! thanks#2023-04-0514:06Dave MaysI'm sure this is already been discussed, but is there a tutorial for publishing Electric programs in general? I was expecting to see a compiled output folder with a JS and a Java file somewhere that you would publish, but haven't found which that might be while poking around some of the starter examples.#2023-04-0514:53Dustin Getz@U04S13KL1R9 https://github.com/hyperfiddle/electric-starter-app has full CI/CD, dockerfile, uberjar, github actions, http://fly.io config#2023-04-0514:55Dustin GetzHere are the instructions: https://github.com/hyperfiddle/electric-starter-app/blob/main/src-build/build.md#2023-04-0514:55Dustin GetzDoes this answer your question?#2023-04-0515:07Dave MaysAmazing thanks!#2023-04-0809:01andersmurphyDocker file worked for https://render.com too, only thing that needed changing was adding a /healthz endpoint to the router that returns a 200.#2023-04-1913:42Dave MaysHas this been reworked? The links above go to a 404 on Github now. Wondering if moved / updated.
Oh looks like this might be it? (But this still refers to "Photon" not electric, so sounds outdated?: https://github.com/hyperfiddle/electric/blob/master/Readme_deploy.md)
Ah this should be it!
https://github.com/hyperfiddle/electric-starter-app/blob/main/README.md#2023-04-1913:48Dustin GetzThose instructions in deploy.md in starter app were moved to the top level readme https://github.com/hyperfiddle/electric-starter-app#2023-04-0423:43vincentbeen away a minute, did y'all post an uberjar tutorial ?#2023-04-0501:02Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/main/src-build/build.clj#L54#2023-04-1122:28vincentso effin cool#2023-04-1123:39vincenti see the code added, pardon the newbiness, is lein uberjar enough?#2023-04-1201:12Dustin Getz i haven’t used lein in years, i personally recommend forking this working repo and the scripts should just work#2023-04-0423:54vincentor maybe dockerfile is the way?#2023-04-0516:00Dave Mayshttps://clojurians.slack.com/archives/C7Q9GSHFV/p1680706428541629?thread_ts=1680649810.480789&cid=C7Q9GSHFV#2023-04-0516:00Dave Mayshttps://clojurians.slack.com/archives/C7Q9GSHFV/p1680706504443239?thread_ts=1680649810.480789&cid=C7Q9GSHFV#2023-04-0611:56arohner(ns contrib.gridsheet
"todo deprecate, use HFQL grid. Used by datomic-browser and folder-explorer"
What/where is HFQL?#2023-04-0612:53Dustin Getz🙂 Revealing this soon! It's a declarative CRUD/hypermedia DSL for back office apps like this, an app we built for our design partner a Series B startup, to operate their mission critical customer success function#2023-04-0612:58Dustin Getz#2023-04-0615:50teodorluThe “big table connected to prod” seems like a good fit for backoffice apps / admin panels, and a good fit for Electric. Looking forward to the release!#2023-04-1020:37mattiasWill HFQL be open source? 😧 #2023-04-1210:34arohnerI have all of musicbrainz locally and I’m trying to load 2.5M rows into contrib.gridsheet and it slows to a crawl. Will HQFL solve that problem, or is there something I should work on regardless?#2023-04-1210:46Dustin Getzlikely on us, iirc the pagination of the resultant is implemented naively, the gridsheet needs to be refactored to let you inject an efficient database-native resultset pagination logic#2023-04-1210:47Dustin Getzhas nothing to do with Electric or HFQL it’s just a bad datomic query#2023-04-1210:48Dustin Getz(->> … (sort-by .) (drop n) (take m))#2023-04-1210:49arohneryeah, I don’t see an obvious place to stick that ‘window’ function in gridsheet#2023-04-1210:49arohnerI’m not using datomic, I’m using a proprietary datomic-alike that I need to OSS#2023-04-1210:49Dustin Getzcan’t wait to see that#2023-04-1210:50Dustin Getzdoes it support sorting and pagination queries?#2023-04-1210:50Dustin Getzthe gridsheet just needs to be refactored, it will take a day #2023-04-1210:51arohnerMostly, yeah. I have datoms and index-range, and both support limit and reverse#2023-04-0614:19arohnerI have a very vague question, sorry. I’m got electric mostly running on an internal app, adapted from electric-starter-app. On the starter app, I’ve replaced the watch on datascript with a watch on a ref of my own database. Via println I can see that the transacts are working, but the page doesn’t reload reactively. Everything renders properly when I hard reload the page. Any ideas on what to check?#2023-04-0614:20Dustin Getzwhat database? datascript conn objects implement the atom protocols#2023-04-0614:20arohnerit’s a proprietary thing I will OSS later this year. the e/watch is on a real clojure ref#2023-04-0614:21arohnerI posted the sample in channel. The println fires as expected#2023-04-0614:21Dustin GetzSee this discussion: https://clojurians.slack.com/archives/C7Q9GSHFV/p1679824993170869#2023-04-0614:19arohner(e/defn Todo-list []
(e/server
(binding [current-tx (e/watch !conn-ref)]
(let [db (f/db !conn)]
(println "current-tx is" current-tx)
(e/client
(dom/link (dom/props {:rel :stylesheet :href "/todo-list.css"}))#2023-04-0614:28xificurCNo new db is getting computed because !conn doesn't change. Electric only re-runs code where inputs change. If !conn-ref updates with each server transaction you can force a dependency on that#2023-04-0614:29arohnerWhat is the boundary of an input? A function?#2023-04-0614:30arohner!conn-ref is a normal clojure ref that changes as expected#2023-04-0614:31xificurCyes. In (f a b c d) , if f already ran once, the result is cached. as long as the inputs a b c d stay clojure.core/= f will not re-run#2023-04-0614:34xificurCso (f/db !conn) would re-run only if !conn changes to a new value that is not = to the previous value. Does that make sense?#2023-04-0614:34arohneryeah, thanks#2023-04-0614:40arohneris something special happening with the binding in the example above? In electric-starter-app, why (e/def db) at all?#2023-04-0614:46xificurCElectric defs are dynamic. In the starter app we use binding on it to bind the db once at the entrypoint and have it available throughout the app without the need to pass it around in lexical scope. You can think of it as lightweight dependency injection.#2023-04-0614:48arohnerput another way, if I replace that binding with a let, (and fix the places where db is used without being passed in), does the app still function?#2023-04-0614:49xificurCYes. Why would you want to pass it in lexical scope though?#2023-04-0614:50arohnerjust checking my understanding. My regular clojure style wouldn’t use binding so I wanted to understand whether it was essential or not ¯\(ツ)/¯#2023-04-0614:51xificurCI see. I share your feeling wrt clojure style. Luckily electric dynamic binding, unlike in clojure, works#2023-04-1210:54arohneraha, the other trick here is that values passed from server to client must be transit-able. binding is a hack around that.#2023-04-1210:58xificurCcurrently yes, but you can consider it tech debt. E.g. in
(e/server
(let [o (Object.)]
(e/client (Foo. o))))
(e/defn Foo [o] (e/server (prn o)))
electric will attempt and fail to transfer o but there is no need for the transfer since the value is only used on the server. A future iteration on the compiler will fix that.#2023-04-0618:15alexdavisHi! started playing with electric and its super awesome and exciting! Couple of quick questions:
• are there any api docs? I am not good at understanding macros and although the tutorials are great, it would be nice to see an exhaustive list of options available and maybe some formal descriptions of how this stuff actually works
• I cloned the example todo app and added a Thread/sleep to the query to simulate a slow db query. However to my surprise this makes anything else that is inside an e/server block wait for that query to finish before it will be sent to the client, even if they aren't children of the component with the slow query. I guess I was expecting something like react server components/remix where each component can be streamed to the client individually? Or am I just missing something?
(e/defn Todo-list []
(e/server
(binding [db (e/watch !conn)]
(e/client
(dom/link (dom/props {:rel :stylesheet :href "/todo-list.css"}))
(dom/div (dom/props {:class "todo-list"})
(dom/div
{:class "should render immediately"}
(e/server
(let [e (d/entity db 1)]
(e/client
(dom/text (e/server (:task/description e)))))))
(dom/div
{:class "should render after 1000ms"}
(e/server
(let [e (do
(Thread/sleep 1000)
(d/entity db 1))]
(e/client
(dom/text (e/server (:task/description e))))))))))))#2023-04-0619:04Dustin GetzUse e/offload to move a slow query to a threadpool and not block the server (mental model here is everything is async so you can't be blocking). We don't document that in the tutorial currently because it has a surprising edge case that we want to improve before everyone starts using it#2023-04-0619:05Dustin Getzso,
(let [e (e/offload #(do (Thread/sleep 1000) (d/entity db 1)))] ... )
this is 100% fine to do and will not block the server
The surprising edge case is that e/offload can sometimes cause expressions underneath it to render too often, which we'd like to fix before recommending this widely#2023-04-0619:06Dustin GetzDocs are coming, should land next week#2023-04-0621:11alexdavisGreat! exciting times 🙂#2023-04-0818:04Dustin GetzElectric v2-263 released — https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md — focused on deployment, hot code reloading fixes, cljs advanced mode fixed, bugfixes. Please upgrade! There are a few breaking changes with action required. If you're running off of master, we recommend you stop doing that as we are doing some git repo reorganization. Also a missionary partial rewrite is incoming, followed by Electric runtime major changes.#2023-04-1013:08tobiasUnder what conditions does the websocket connection timeout?
I tried giving each websocket session its own atom like this
;; Each websocket session gets its own atom. State not shared between browser tabs
(e/def !unshared-server-state #?(:clj (atom {})
:cljs nil))
I noticed that if I leave my computer for 30 min and come back then the server atom gets reset to its initial (empty) value. Is this intended behaviour?
Basically what I want is for each user to get their own server state (i.e. not multiplayer) but I don't want them to lose that state if they leave their computer and come back after lunch break. What's the best way to do that? This is for a very simple back office app with no database and no authentication. Right now as a workaround I'm storing all state on the client but I'd like to understand how I could do it with state on the server.
Maybe I should be doing something like using a shared server atom and giving every user their own key inside it based on a random cookie I assign them?#2023-04-1013:36Dustin GetzLet me split apart the questions:
1. how to do per-user state in electric?
2. how to make per-user state durable?
3. under what conditions is the connection lost?#2023-04-1013:38Dustin Getz(1) see the atom in the https://electric-examples-app.fly.dev/user.demo-system-properties!SystemProperties L17. In Electric, all local state is tied to the lifecycle of the ws connection (and isolated to that connection) with the exception of global state#2023-04-1013:42Dustin Getz(3) (a) connection is terminated gracefully when browser tab is closed; (b) connection is lost after 60 seconds without heartbeat (i.e. lost internet service, or went to another tab*), (c) connection is lost on server restart (e.g. deploy)
*If you blur the tab, everything pauses (because we schedule all work with requestAnimationFrame) including consumption of the websocket (everything is backpressured).
Connection is re-established when (a) the tab is focused again, (b) the auto-retry logic sucessfully reconnects. On reconnection, today the client reboots, losing local state. (We know how to implement resync on reconnect without client reboot, it's on the roadmap)#2023-04-1013:43Dustin Getz(2) you'll need to write code for this, e.g. store it in redis, localstorage, cookie etc#2023-04-1013:45tobiasthat makes sense, thanks for explaining#2023-04-1013:47Dustin Getzedited#2023-04-1013:48tobiassystem properties tutorial for other people reading this (link earlier went to local host for me)
https://electric-examples-app.fly.dev/user.demo-system-properties!SystemProperties#2023-04-1013:52Dustin Getzthanks, fixed link#2023-04-1013:21tobiasAnother question: My app has a slow operation (hitting some apis) that takes ~5 seconds when the user hits the "submit" button. What's the best way to show a spinner in the waiting period? I notice that ui/button pretty clever in that it automatically applies disabled and aria-busy="true" to the button during waiting, but is there also something I could do to conditionally display a dom element during that waiting period?#2023-04-1013:33Dustin Getzthe next tutorial shows how to do control load states, i'll try to publish it today#2023-04-1013:34Dustin Getzit's exactly this: https://github.com/hyperfiddle/electric/blob/a1907db5b77931f777205b0fb6b15722154435d8/src-docs/user/demo_chat.cljc#L31-L32 , the tutorial just explains it#2023-04-1013:37tobiasAwesome, thanks! I also just noticed on-pending in another demo
https://gist.github.com/dustingetz/2c1916766be8a61baa39f9f88feafc44
Is one way preferred over the other?#2023-04-1013:49Dustin Getzyeah, on-pending is just a macro over the try/catch Pending pattern#2023-04-1113:46Dustin GetzFYI: I just pushed breaking changes to master without documentation. Reminder to please get off of master and use the https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#2023-04-1114:10teodorluI did a quick check for dependency behavior with #babashka-neil.
TL;DR:
• neil dep add com.hyperfiddle/electric installs version v2-alpha-263-g89da9d11
• running neil dep add com.hyperfiddle/electric again when an old clojars-alpha version is installed, installs the latest clojars alpha version
---
Details:
1. neil dep add com.hyperfiddle/electric installs v2-alpha-263-g89da9d11
2. but neil dep upgrade com.hyperfiddle/electric doesn’t seem to be able to update an old electic version to the new version.
3. running neil dep add com.hyperfiddle/electric again when an alpha version is installed installs the new version.
neil 0.1.60 changed the default dep add behavior to add the latest unstable version of no stable versions are found. But nep upgrade hasn’t gotten the same behavior.
So using neil dep add seems to give the latest recommended alpha version.#2023-04-1114:31Dustin Getzwhat?#2023-04-1116:34teodorluoh, sorry, that probably read like it was totally out of context. i was thinking that the recent change in neil behavior might be useful for electric users, since it matches which electric version you want people to use.#2023-04-1116:37Dustin Getzwhat is neil?#2023-04-1116:40teodorluNeil is a babashka CLI for installing and updating Clojure dependencies in deps.edn projects:
https://github.com/babashka/neil#2023-04-1122:29vincentOkay so I'm working from the XTDB-Electric examplez#2023-04-1122:30vincentI'm wondering, how I would do a cloud deploy like electric has for the example pages w/ 5 worldwide mirros#2023-04-1200:43Dustin Getzyou’ll need a cross region cloud database like say google spanner not xtdb - it’s not trivial#2023-04-1201:19vincentfack x)#2023-04-1209:01alexdavisYou can use a single central document/tx store for xtdb and deploy the electric instances to the edge. xtdb will store the document and tx logs on each node so you would only have to go to the central db for writes or cold reads. If you're using http://fly.io you should be able to just deploy a single region postgres and use that. you can get help on setting up your xtdb config in #CG3AM2F7V, you'll probably want to utilise checkpointing or the electric nodes will be slow to start#2023-04-1210:33Dustin Getz@U7KPK060K please say more and/or ELI5 without XTDB prior knowledge and i will add this to our docs#2023-04-1211:04alexdavisI'm far from an expert on XTDB so @U899JBRPF correct me if I'm wrong about any of this, but here are the basics as I understand them:
• an XTDB deployment consists of one or more Nodes, a document store and a tx store (immutable transaction log of changes to the doc store). You can choose different places to persist these, by default I think they both get stored on the node using rocksdb, but you can also persist them outside the node using one of thehttps://docs.xtdb.com/storage/
• the XTDB nodes usually run in the same process as the host application (in this case electric) and each node connects (remotely via http/jdbc etc or in memory/local disc) to the doc/tx store
• when a node starts up it will retrieve a copy of the indexes and store them in memory. queries made on the node (e.g using xt/q or xt/entity) will be executed against these local indexes (though I'm a little unclear about this because I know you can configure the amount of memory available to xtdb for local indexes, so possibly there are cases where the query will have to wait for the node to fetch data from the central store).
• Deploying electric to the edge is probably beneficial if the app isn't write heavy, as each edge node will effectively cache the db and trips to the central store will only need to happen when doing writes or reading recently stored data, but electric does encourage frequent writes for realtime applications and in those cases it might be better to have the electric node close to the doc/tx store
There is more info available here (and in the guides/tutorials on the same site) https://docs.xtdb.com/concepts/what-is-xtdb/#2023-04-1922:33vincentI just figured XTDB is a drop-in and free replacement for Datomic, is this assumption flawed#2023-04-1922:33vincentMaybe I can just go with Datomic Free, and this will be simpler significantly#2023-04-1922:38Dustin Getzdatomic free is probably the easiest possible thing#2023-04-1922:40Dustin GetzIt's still not quite trivial with docker, you'll need to configure a persistent disk for the state if you haven't used Docker before. (IIRC, i can barely keep the container stuff straight) The dead simple thing is to use digital ocean, ssh in, mount persistent disk from digital ocean admin panel, install datomic, run app#2023-04-1922:41Dustin GetzOnce you outgrow that you can dockerize it#2023-04-1923:24vincentOkay cool. So i can copy the electric-starter-app to my server, link in datomic, and should be good to go?#2023-04-2000:37Dustin Getzyeah, make sure the hard disk doesn’t get erased when the box restarts#2023-04-1122:30vincentI usually log into a box and run a server from a specific ip. cloud deploys are a little different and i was wondering if anyone had any pointers#2023-04-1200:48Dustin Getzdo you need cloud? we need it because our demos are accessed worldwide; digital ocean by ssh is perfectly acceptable#2023-04-1200:49Dustin Getzyou can also look into railway, https://render.app, google cloud run - these all involve learning docker, which with XTDB may not be trivial#2023-04-1200:50Dustin Getzreally figuring how how to run XTDB in prod is the first thing to do, this will determine your options for hosting #2023-04-1201:20vincentI see, I don't need cloud. I just thought it was the next logical step. But if db is whack++ then I can just do a singular local instance based in NYC area or whatever the digital ocean cluster is#2023-04-1122:33vincent
Also, unrelatedly to deployment,
I have a question regarding views. I have a dom/button with the on-click set to toggle an atom, and an if statement in the view (or a case statement) but the view never changes.
I'm curious what I would be doing incorrectly there#2023-04-1200:45Dustin Getzpost the snippet, did you forget e/watch?#2023-04-1201:21vincentpossibly! ty for the idea. will check.#2023-04-1222:41fabraoThis reminding me the Java GWT, it's similar that you can write in only one place, and it compiles for server and client, but much less complex.#2023-04-1308:27arohnerI’m still kind of confused when views are redrawn. I’m trying to extend gridsheet to make the column header clickable to change the sort order. I have
(let [!sort (e/watch (atom {}))
rows (->> (q '{:find ...}) (sort-by !sort)]
The on click handler is working, and my !sort atom is being updated, but then the query on the next line depending on !sort doesn’t refresh. Does everything reactive need to be in missionary?#2023-04-1309:26xificurCIf !sort changes sort-by will re-run. You can check if it changed with a simple (println !sort). If that's working then what's failing is some other piece of code you haven't shared.
BTW our usage of prefixing with a ! is for mutable references, in your example you're e/watching over it so no need for the bang. Typically it's (let [!sort (atom {}), sort (e/watch !sort)] ...).
If you can show a bigger sample of your code where
• the update to the atom happens
• the part of the view you want to refresh doesn't
we should be able to figure this out. Grabbing a call is also an option#2023-04-1309:28xificurC> Does everything reactive need to be in missionary?
Yes. e/watch returns a missionary flow. But you don't need to construct low-level missionary flows in 90% of your app, only on your edges of the system. So we wrap atoms, dom events, databases etc, but if you have your own database you might need some missionary code around that.#2023-04-1309:44arohnerThanks. Theoretically I’m on a vacation this week, but if I’m still having trouble next week a call would be nice#2023-04-1309:45arohnerin my edited gridsheet.cljc:
(dom/div (dom/props {:role "columnheader"
:style {:position "sticky" #_"fixed" :top (str 0 "px")
:background-color "rgb(248 250 252)" :box-shadow "0 1px gray"}})
(dom/on "click" (e/fn [e]
(when header-on-click
(new (header-on-click e k)))
nil))
(dom/text (name k)))#2023-04-1309:45arohner::header-on-click (e/fn [e col]
(e/server
(println "click" col)
(reset! sort {col true})
(println "sort:" sort)))#2023-04-1309:45arohnerclicking gives me ‘unserializable reference transfer’#2023-04-1309:50xificurCthe click e/fn takes e as an arg, so that must be OK. header-on-click and k are free variables, so either header-on-click is defined in an e/server block or k is something that cannot be serialized. You can try stubbing/commenting them out one by one to see which one is the culprit#2023-04-1309:51arohneryes, I am in server context when defining ::header-on-click. so that needs to be (e/client (e/fn ...))?#2023-04-1309:53xificurCyes that should work. Electric functions aren't serializable right now. In the future your code should work without the e/client wrapper#2023-04-1309:53arohneraha#2023-04-1309:54arohner(e/client (e/fn []) returns nil on the server side though. I was trying to pass the on-click via props. How do I work around that?#2023-04-1310:01xificurChard to say without seeing the actual code. 1 thing that comes to mind is turning your e/defn into a macro. Right now e/fn boundaries transfer their variables upon calling them (a past mistake we'll be fixing). Switching to a macro solves the coloring problem for now
(e/defn Foo [F] (e/server (new F)))
(Foo. (e/fn [] 1))) ; breaks, tries to transfer
(Foo. (e/server (e/fn [] 1))) ; still breaks
(defmacro foo [F] `(e/server (new ~F)))
(foo (e/fn [] 1)) ; works
(foo (e/server (e/fn [] 1))) ; works too
#2023-04-1310:02arohnergot it, thanks for your help#2023-04-1310:02xificurCnp, let us know if you need more help#2023-04-1309:21arohnerI’m also unclear on what new is doing#2023-04-1309:31xificurCelectric functions are called with new . Since electric functions are missionary flows, missionary flows are also called with it. From another POV it's the monadic join#2023-04-1309:32xificurC• (new Foo 1)
• (Foo. 1)
• (new (something-returning-a-missionary-flow))#2023-04-1310:17Dustin Getznew is now explained here: https://electric-examples-app.fly.dev/user.tutorial-lifecycle!%4Cifecycle#2023-04-1310:17Dustin Getztutorials are recently updated , hadn’t announced them yet#2023-04-1310:18Dustin Getzerm that is not the latest version of that tutorial, i rewrote it yesterday. i’ll deploy in an hour#2023-04-1310:38Adrian SmithWith the directory explorer demo
Scrolling through the list fast the file names feel glitchy and unstable
I think my assumption is that I’m looking at a stable list of things and the scrollbox is my viewport into that stable thing
but what I’m seeing is the flicker of the intermediate steps of the scroll, which I assume trigger the IO/repaint of every row
I wonder if overfetching the view slightly so the IO is slightly back loaded would help take out the flicker?#2023-04-1310:54Dustin GetzYou're right, there's an issue, we haven't diagnosed it. I think it's related to the scroll math (i.e. rounding error) not IO, if you scroll one record at a time it often ticks twice in a row causing some sort of double-render or something for each scroll tick. It's much worse when hyperlinks are visible on the screen, if you scroll way down the list the issue is much less visible#2023-04-1310:55Dustin GetzYou're also right that overfetching will make it even faster (especially for slow backends), we'll take a look at it when we rewrite this#2023-04-1310:59Dustin GetzThe latest tutorials are deployed, here's the rewritten https://electric-examples-app.fly.dev/user.tutorial-lifecycle!Lifecycle#2023-04-1315:35fabraoSo, the fact of if is false means the component is destroyed?#2023-04-1312:35markaddlemanI was looking at the component lifecycle tutorial at https://electric-examples-app.fly.dev/user.tutorial-lifecycle!Lifecycle and became mesmerized at the blinking for a few minutes. Then, the page went blank and the console reporting the attached server crash message#2023-04-1315:57Dustin Getzthanks, we will look into it#2023-04-1315:57Dustin GetzIt recovered i assume?#2023-04-1315:57markaddlemanYeah, after a refresh#2023-04-1320:40fabraodo you think can I use Eletric in production enviroment?#2023-04-1320:45Dustin Getzyes, main risk would be not the electric core tech but any database integrations#2023-04-1320:45Dustin GetzWe've historically focused on Datomic, though obviously Electric supports any data backend#2023-04-1320:47fabraoYes, I'm thinking using libraries that will listen database notification, as we have in Hasura, to provide a way to broadcast the changes to clients.#2023-04-1320:48Dustin GetzThat part will need an adapter to Electric coded in Missionary#2023-04-1320:49fabraobut if you provide an atom as the shared repository to when it's changed, it will automatic broadcast it?#2023-04-1320:50Dustin GetzYes, if you give us a type that implements the proper Atom protocols, e/watch will work#2023-04-1320:52fabraoGreat !!! About the security, how can I trust the intercomunication from client to server?#2023-04-1320:53Dustin Getzwebsocket is over https assuming the application is also served https#2023-04-1320:55fabraoso we have to provide the same ways for authentication for secure system?#2023-04-1320:55Dustin GetzYeah, here's an example with auth through a cookie: https://electric-examples-app.fly.dev/user.demo-chat-extended!ChatExtended#2023-04-1320:57fabraogot it, thank you.#2023-04-1321:06fabraoI was studing Eletric yesterday and I saw this https://github.com/yjs/yjs for comparing the complexity , but Eletric do almost the same in more elegant way :rolling_on_the_floor_laughing:#2023-04-1321:31Dustin GetzElectric and CRDTs should be compatible, there are times when you need CRDT semantics and Electric as a language lets you just do that.#2023-04-1320:57Dustin GetzTutorials - more have been written, tutorials 4+ are new : https://electric-examples-app.fly.dev/user.demo-chat!Chat#2023-04-1405:49tobiasJust curious: can Electric be used for purely client-side apps, as a sort of Reagent alternative? I'm guessing not as the client would try to establish a websocket connection with the server?
I have no particular use case for this, just interested in understanding what's possible.#2023-04-1406:55xificurCSure! We don't have an example of that, it would require a different entrypoint#2023-04-1408:38tobiasThat's neat. I remember a few years ago there was talk of making something like Reagent that didn't rely on React (e.g. Mr Clean https://www.reddit.com/r/Clojure/comments/abc4pi/mr_clean_is_a_reagent_compatible_clojurescript/), and it seems like you guys have created that as part of Electric. I understand that the goals of Electric are much larger of course.#2023-04-1408:56xificurCthanks for the link. Seems to be a reagent compatible library swapping out react.
Electric is a reactive language. What this means is the DOM stuff is ~300loc with some deprecated stuff / tech debt. But we could do the same with any other UI toolkit, e.g. Swing or JavaFX. The loc would be similar.#2023-04-1409:58tobiasWow that's cool that the UI layer can in principle be swapped out for something else. I'm still wrapping my head around the reactive language idea.#2023-04-1414:03chromalchemyI understand Electric is fine-grained across the network boundary in a sense that functions that don’t need to be re-run are not (given the omniscience of the cross-boundary DAG).
I am curious about the nature of reactivity when focused on just the client/Dom state.
@U0PUGPSFR #matrix Talks of “fine-grained”-ness in terms of reactivity operating at the property/attribute level (rather than view function) of a UI component. It also maintains a Dag to achieve omniscience, enabling next level developer UX.
https://github.com/kennytilton/matrix/wiki/introduction#property-oriented-data-flow
Heres an interesting discussion on some of these dynamics.
https://clojurians.slack.com/archives/CKCBP3QF9/p1677614613185429
This stuff appeals to me for wrangling view design dynamics (as opposed to focus on cross network state).
Dustin mentioned Electric offers local reactive semantics similar to solid.js in this respect. I am not very familiar with it. Does Electric/solid.js encompass this property-level granularity? Or is it a different scope or approach?
Or (just thinking out lout)… If the meaningful state for reaction resides on the server (and Electric enables using this fabulously), maybe this comprehensive client-only reactivity is not a fully applicable in this case?
Either way, from the above I assume there might not be problem to add something like Matrix to cljs side for extra client semantics (even if it is macro heavy and probably has some overlapping concerns..)?#2023-04-1414:25Dustin Getzfocusing on just dom rendering: both matrix and electric promise fine-grained reactivity and the ability to express maximum possible granularity at the quanta of a single point in the AST (attribute, expression, whatever)#2023-04-1414:26Dustin GetzSolid IIUC contains implementation mistakes but in concept is similar#2023-04-1415:13chromalchemyCool. I dont want to miss out on any quanta 🤓#2023-04-1415:13chromalchemyI had the wrong discussion link earlier. Here is the right one
https://clojurians.slack.com/archives/CKCBP3QF9/p1677614613185429#2023-04-1415:06andersmurphySo I’m trying to render something else when there’s a Failure. i.e show the NewGame Screen instead of GameScreen if for whatever reason there’s an error updating state. Currently this doesn’t seem to work. Any ideas?
(try
(e/server
(swap! !state add-session-to-presence game-id session-id)
(e/on-unmount
#(swap! !state remove-session-from-presence
game-id session-id)))
(e/client (GameScreen. game-id))
(catch Failure _
(e/client (NewGameScreen.))))#2023-04-1415:19Dustin Getzwhat is Failure#2023-04-1415:30andersmurphyhttps://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric.cljc#L26-L31#2023-04-1415:30xificurCif it's hyperfiddle.electric.Failure that's an internal type you'll never see in userspace. If you want a catch-all you need Throwable on the server and :default on the client#2023-04-1415:36andersmurphySo those swap! can throw IllegalStateException so something like this?
(e/server
(try
(e/server
(swap! !state add-session-to-presence game-id session-id)
(e/on-unmount
#(swap! !state remove-session-from-presence
game-id session-id)))
(e/client (GameScreen. game-id))
(catch :default _
(e/client (NewGameScreen.)))))#2023-04-1415:36andersmurphyThat renders both the GameScreen and NewGameScreen components#2023-04-1415:38andersmurphyI guess I’m expecting try catch to behave like it does on Clojure i.e stop execution the moment something fails. I understand that’s not how it works in Electric. Is there a way to bring that back? Or is there an electric convention for this?#2023-04-1415:38xificurCindeed, that's how try/catch works, a catch handler doesn't unmount the try body, because an exception can resolve later, which would cause flicker#2023-04-1415:39Dustin Getzthe catch here is on the server so catch Exception or IllegalStateException#2023-04-1415:40Dustin GetzThe https://electric-examples-app.fly.dev/user.demo-chat!Chat has some explanation of reactive exceptions#2023-04-1415:42xificurCI don't know what the 2 components do/look like, but off the top of my head you could reset! an atom and if on that
(let [!threw? (atom false), threw? (e/watch !threw?)]
(try (if threw? (NewGameScreen.) (case (GameScreen. game-id) (reset! !threw? false)))
(catch :default _ (reset! !threw? true)))) #2023-04-1415:44Dustin Getz> behave like it does on Clojure i.e stop execution the moment something fails
> Is there a way to bring that back?
just don't catch the exception might do what you want#2023-04-1415:44xificurCcan't NewGameScreen be a modal popup that unmounts after pressing something?#2023-04-1415:51andersmurphyI guess I’m modeling it as screens cause that’s similar to how we do things on our non electric mobile first stuff. I could probably hack it with a modal. But conceptually this is similar to routing. Basically If error show A otherwise B seems like something quite common?
@U09K620SG That’s the tutorial I was working off (they are super helpful).
@U09FL65DK I’ll try that atom for tracking the exception, feels a bit clunky compared to how easy everything else is in electric.#2023-04-1415:56Dustin GetzThere is surely a cleaner way to do this, i will try to find a good example of routing#2023-04-1415:56Dustin Getzwhy do you want to use throwing an exception to signal a route change?#2023-04-1415:58andersmurphyKinda like a redirect conceptually I guess. I’ve been using the contrib goog.history stuff and for the most part that’s been fine.#2023-04-1415:58andersmurphyMaybe, I’m still shackled to rest/http concepts#2023-04-1415:59Dustin Getzto me, conceptually, clicking a hyperlink (click callback) or setting document location all map conceptually to reset! on an atom#2023-04-1415:59andersmurphySo this isn’t at the route, it’s in a parent component. But on error I want to show a different component.#2023-04-1416:01andersmurphy(Game (if success ShowGame ShowCreateNewGame))#2023-04-1416:02andersmurphy(if (e/server ;; return false on error
)
(GameScreen.)
(NewGameScreen.))#2023-04-1416:04Dustin Getz> On error I want to show a different component
in that case, this seems appropriate to me, as it's the exception that represents a request for state change
(catch Exception _ (reset! !route `NewGameScreen))
#2023-04-1416:06xificurCIndeed, you're asking for a router#2023-04-1416:07andersmurphyRight, but what about at a more granular level. Show button A when error, show button B when not error. That’s not really any different?#2023-04-1416:10andersmurphyYou wouldn’t use routes for that but conceptually it shows A or B. Right?#2023-04-1416:12andersmurphyBtw as an aside I’m absolutely loving electric, it makes a lot of really hard things easy. I also appreciate the help.#2023-04-1416:19Dustin Getzthis is a bit meta but i would question the essential difference between single-state-atom and document.location. for example we have an experimental router that looks like an atom but puts state into the route#2023-04-1416:20Dustin Getzso imo the decision of whether to put state in the route or in an atom is arbitrary, do what feels nice to you in your app#2023-04-1416:55xificurC> Show button A when error, show button B when not error
>
Errors usually add a button. There's also another common pattern
(try (case (foo) (Ok.)) (catch :default e (Error. e)))
#2023-04-1507:12andersmurphyhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1681488258907999?thread_ts=1681484777.404339&cid=C7Q9GSHFV
Using an atom as a location pointer for routing is really interesting. Is there a reason you are passing the symbol?
I’ve tried (catch Exception _ (reset! !route NewGameScreen)) without the quote and it means your top level routing doesn’t need a case lookup. Now the top level of the app looks super simple and doesn’t need to change when new (internal) routes are added:
(e/defn Game []
(e/client
(dom/link (dom/props {:rel :stylesheet :href "/styles.css"}))
(if route
(route.)
(NewGameScreen.))))
NewGameScreen in this case is whatever you want you’re home/starting screen to be. Obviously, this isn’t much good for url/path based external links and you’d need some sort of look up for that (I’m sure there are other limitations I haven’t considered). But for internal “screen” changes this seems to work quite nicely. Thanks again for all the help!#2023-04-1507:29andersmurphyI don’t quite understand the use of case with two arguments like that? Is it being used as a when / and.
https://clojurians.slack.com/archives/C7Q9GSHFV/p1681491343538329?thread_ts=1681484777.404339&cid=C7Q9GSHFV#2023-04-1507:55xificurCThe case ensures the second arg runs only if the first one didn't throw an exception#2023-04-1508:14andersmurphyWould it behave differently (in electric) to (when (foo) (Ok.)) or (and (foo) (Ok.))assuming (foo) returns truethy on success?#2023-04-1508:17xificurCIt would behave the same. case makes no assumption about the return value#2023-04-1420:09Dag NorbergMaybe I’m missing something basic but does Electric support variadic functions?
This is what I’m trying to do
(e/defn Debugger [& xs]
(dom/div
(dom/text (str xs))))
(e/defn App []
(Debugger. watcher-1 watcher-2 watcher-3))
#2023-04-1420:11Dag NorbergObviously you could just pass a vector but still#2023-04-1420:51Dustin GetzNot yet, sorry about the confusion#2023-04-1421:01Dag Norbergok thanks for clarifying!#2023-04-1422:56tobiasThis gist for working with promises works great:
https://gist.github.com/dustingetz/8823e47c13f780d18938363d1d641b5b
But why doesn't it work if I try to put the (new (e/task->cp ...)) inside the function definition?
(defn await-promise "Returns a task completing with the result of given promise"
[p]
(let [v (m/dfv)]
(.then p
#(v (fn [] %))
#(v (fn [] (throw %))))
(m/absolve v)))
(e/defn compact-await-promise
[p]
(new (e/task->cp (await-promise p))))
(e/defn Todo-list []
(dom/p (dom/text "Promise value using original method: ")
;; This displays "hi" as expected:
(dom/text (new (e/task->cp (await-promise (.resolve js/Promise "hi"))))))
(dom/p (dom/text "Promise value using compact function: ")
;; This displays nothing
(dom/text (compact-await-promise (.resolve js/Promise "hi")))))#2023-04-1503:01noonianSince compact-await-promise is an electric function, you need to call it with new e.g. (new (compact-await-promise ...)) or (compact-await-promise. ...) using dot notation. Does it work after making that change?#2023-04-1503:49tobiasThat works, thank you!
(e/defn Todo-list []
(dom/p (dom/text "Promise value using original method: ")
;; This displays "hi" as expected:
(dom/text (new (e/task->cp (await-promise (.resolve js/Promise "hi"))))))
(dom/p (dom/text "Promise value using compact function: ")
;; This displays nothing
(dom/text (compact-await-promise (.resolve js/Promise "hi"))))
(dom/p (dom/text "Promise value using compact function: ")
;; This displays "hi" as expected. Note the dot after compact-await-promise
(dom/text (compact-await-promise. (.resolve js/Promise "hi")))))#2023-04-1511:28Dustin Getz@U052PH695 minor tweak for your understanding -
• (Compact-await-promise. ...) -- correct
• (new (Compact-await-promise ...)) -- incorrect
• (new Compact-await-promise ...) -- correct
I've also capitalized the electric fns to help make clear that capitalized things are called with new (as if a class)#2023-04-1511:28Dustin GetzSee the recently published https://electric-examples-app.fly.dev/user.tutorial-lifecycle!Lifecycle which describes new and how electric fns have object-like aspects#2023-04-1517:24noonianThank you. My bad on the incorrect syntax in the (new (Fn ..)) example.
As far as capitalization goes, that is just a convention to help differentiate between Clojure and Electric functions right? I.e. if you don't capitalize the function name that doesn't change how electric interprets the code?#2023-04-1517:38Dustin Getzcorrect#2023-04-1517:38Dustin Getzit also helps you remember where the new goes, because (new (f x)) can be a thing too if (f x) returns a missionary flow#2023-04-1507:19maxtI was surprised that this doesn’t work:
(Section. "Contact"
[(Input. "Name")
(Input. "Address")])
It seems like you can’t pass components as values. Indeed they seem to be nil when printed (println (Input. "Name)))
Obviously I fail to understand what really happens, and that bothers me. Shouldn’t I think of instantiated e/defns as values?#2023-04-1507:52xificurCElectric functions return values. What is Input? What do you expect your code to do?#2023-04-1508:04maxtInput and Section are defined as follows:
(e/defn Input [label name]
(dom/label (dom/props {:for name})
(dom/text label))
(ui/input (dom/props {:id name}) ""
(e/fn [v])))
(e/defn Section [header components]
(dom/h2 (dom/text header))
(dom/div
(dom/props {:class "input-group"})
(e/for [c components] c)))
I was expecting <label> and <input> to be nested under <div class=“input-group”>.#2023-04-1508:10maxtHere’s a minimal example
(ns app.nesting
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
(e/defn Input [label name]
(dom/label (dom/props {:for name})
(dom/text label))
(ui/input (dom/props {:id name}) ""
(e/fn [v])))
(e/defn Section [header components]
(dom/h2 (dom/text header))
(dom/div
(dom/props {:class "input-group"})
(e/for [c components] c)))
(e/defn Nesting []
(Section. "Contact"
[(Input. "Name" "name")
(Input. "Address" "address")]))
It generates the following HTML:
<label for="name">Name</label><input id="name">
<label for="address">Address</label><input id="address">
<h2>Contact</h2>
<div class="input-group"></div>
#2023-04-1508:14maxtThe part that I don’t understand is that even if I don’t use components in Section, they are still added to the dom.#2023-04-1508:15maxtAlso, if I render them in Section with (dom/text header (str components)) I get [nil nil] which I why I thought they didn’t return a value.#2023-04-1508:22xificurCThey are added to the DOM because you already invoked them. (Input. ...), the dot invokes Input#2023-04-1508:23xificurCYou're not passing in the components but their return values#2023-04-1508:24xificurCI'd make section a macro with & body#2023-04-1508:25xificurCOr you pass them in as (e/fn [] (Input. "Foo"))) and call them in the e/for with new#2023-04-1508:42maxtOk, thank you. Those are good options.
What does new do? How does it now where to add it?#2023-04-1509:16xificurCElectric functions are called with new, or with dot, like jvm interop. (Foo.) is same as (new Foo)#2023-04-1509:27maxtYes, but what happens behind the scenes when you do?#2023-04-1509:36xificurCWhat is behind-the-scenes in this case?#2023-04-1509:39xificurCdom/div is a macro that expands to electric code which will mount a DOM div node under the current parent and set itself as the parent for the rest of its body. Does that answer your question?#2023-04-1509:43maxtYes I think it does. At least I then know when they get added and what is considered the parent.#2023-04-1509:45maxtI tried rewriting it using a macro, but I can’t seem to get that to work either:
(defmacro section [header & components]
`(do
(dom/h2 (dom/text ~header))
(dom/div
(dom/props {:class "input-group"})
~@components)))
(e/defn Nesting []
(section "Contact"
(Input. "Name" "name")
(Input. "Address" "address")))
That fails with Can't take value of macro app.nesting/section#2023-04-1509:47maxtIf I replace the body of Nesting with the macroexpanded version it works#2023-04-1509:55maxt(and sorry if I’m rude, I’m just frustrated that I don’t understand, but I really appreciate that you are trying to help me)#2023-04-1510:04xificurCProbably just a missing require-macros in the ns declaration#2023-04-1510:04xificurC#?(:cljs (:require-macros my-ns-here)))#2023-04-1510:05xificurCHope I got that right, typing on phone :)#2023-04-1510:06xificurCYou're not rude at all, happy to help!#2023-04-1510:08maxtThat worked perfectly! I completely forgot about that part of ClojureScript.#2023-04-1510:41Dustin Getzi’ll write a tutorial for this, thanks for the questions. the key idea is that dom is updated by point side effect. those point writes are orchestrated by electric/missionary in a strongly controlled and deterministic order that can be relied upon. Q: isn’t this imperative? A: sorta kinda not really, this is declarative with respect to network and declarative with respect to reactive update order. However, as you just encountered you do have to know some subtle rules which i will write out in the tutorial.#2023-04-1510:43Dustin Getzone rule is:, arguments to a e/fn are evaluated before being passed in (same as Clojure, imagine they were println rather than dom effects), and this parents the dom elements in a surprising way until you understand what’s happening#2023-04-1510:45Dustin Getzespecially when compared to what a virtual DOM approach would do. a benefit of the point write approach over vdom is a function’s return channel is available to return reactive values rather than DOM descriptions#2023-04-1510:47Dustin Getzplus much faster performance of course, vdom diffing/reconciling is heavy (with even more complex rules for programmer to remember like useEffect) and point writes are light and immediate and no need for useEffect#2023-04-1520:00maxtThank you, that makes the reasoning more understandable!#2023-04-1508:54Dag NorbergDo you have an example how to do recursion in an electric component? E.g. a a simple nested structure with elements that expand when clicked? Have tried different things but the call stack seems to blow up even with small structures#2023-04-1509:17xificurCDo you have an example you can share that blew up the stack?#2023-04-1510:24Dustin Getzi don’t think the electric compiler handles direct recursion yet, iirc. There is a tree view component in the main repo https://github.com/hyperfiddle/electric/blob/master/scratch/photon_tree.cljc (needs repairs) which shows a workaround, also some documentation https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_recursion.cljc#2023-04-1511:25Dustin Getzedited with links#2023-04-1512:45Dag Norberg1.
Two examples I have tried:
(e/defn Tree [coll]
(dom/ul
(e/for [x coll]
(dom/li
(if (coll? x)
(Tree. x)
(dom/text (str x)))))))
(e/defn Tree [coll]
(dom/ul
(e/for [x coll]
(dom/li
(if-not (coll? x)
(dom/text (str x))
(recur x))))))
2.
Thanks, how come it only works using dynamic bindings?#2023-04-1514:47Dustin GetzWe haven't gotten to implementing this in the compiler yet, the binding way is actually likely the low level equivalent that the compiler would emit#2023-04-1514:50Dustin GetzAlso last we looked at this, there was some controversy in the team over what the precise semantics should be, so we wanted to delay for more information#2023-04-1514:59xificurC2 notes:
• You can't recur from a for loop. This is true in Clojure as well
• Electric doesn't have TCO. It doesn't seem to make sense in a graph#2023-04-1515:14Dustin Getz@U09FL65DK is it implemented?#2023-04-1515:30xificurCrecur in e/(de)fn is implemented. Named self recursion not sure, likely not#2023-04-1517:00Dag Norbergthanks again!
just to explain the use case: it would be really cool to have an “inspector” view of the entire application state when developing
it seems like photon-tree would be perfect for this but the syntax needs to be updated?#2023-04-1520:53Dustin Getzyeah photon-tree is unfinished. you’re right about the inspector widget, we’d love that too#2023-04-1516:12Dustin GetzHere's the Hyperfiddle long term goal, aligned to the recenthttps://a16z.com/2023/04/14/the-modern-transactional-stack/#2023-04-1609:49tobiasIs there a way to get Cursive "jump to definition" working on an Electric function when that function is e/defn'd in another namespace?
Example:
(ns app.greet
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]))
(e/defn Hello []
(dom/p (dom/text "Hello World")))
(ns app.todo-list
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
[app.greet :as greet]))
(e/defn Todo-list []
(e/client (greet/Hello.)))
Usually I could click greet/Hello. to jump to it's definition, but it's not working. Instead I see an error message "greet/Hello. cannot be resolved".
If I use (new greet/Hello) then jump to definition works as usual. Maybe I need to give a type hint to Cursive or something so that it knows that Hello is an object?#2023-04-1613:37Dustin GetzThis is a Cursive issue, please ask in #cursive - colin said he's going to add Electric support#2023-04-1613:37Dustin GetzIt works in Calva btw#2023-04-1615:13andersmurphyHas anyone figured out how to not get clj-kondo warnings in e/def and e/defn when referencing cljs functions/values (clj functions/values are fine as that’s the default in cljc)?#2023-04-1615:14borkdude@U0JEEGD4N I think someone in #CHY97NXE2 has asked about this recently#2023-04-1615:14borkdudehttps://clojurians.slack.com/archives/CHY97NXE2/p1681376945593859#2023-04-1615:14borkdudenot exactly e/def, but e/for-by#2023-04-1615:15andersmurphyThanks!#2023-04-1615:16borkdudeIf the syntax is the same as the original you can use :lint-as
{:lint-as
It seems this is already done in some starter apps:
https://github.com/hyperfiddle/electric-starter-app/blob/main/.clj-kondo/config.edn#2023-04-1615:16borkdudebut it's best if electric exported this configuration so everyone gets it automatically#2023-04-1615:16andersmurphyYeah, so this is slightly different. As it’s more about treating the code in e/defn as both clj and cljs when in a cljc file. If that makes sense?#2023-04-1615:17borkdudeIf you are in a .cljc file then everything that is not in a reader conditional is treated as for both clj and cljs#2023-04-1615:18andersmurphyRight, I guess I’m getting lint warnings for things that are defined using the cljs reader and then being referenced. let me get a screenshot.#2023-04-1615:19andersmurphy#2023-04-1615:19andersmurphySo the current hack is to just define !route for both#2023-04-1615:20borkdudeif you define route only for cljs, then I think the red thing is expected?#2023-04-1615:20borkdudeoh, e/client is only client side?#2023-04-1615:21andersmurphyWhat might be possible I guess is to have a hook for e/client that treats the body as cljs#2023-04-1615:22andersmurphyand e/server that treats it as clj#2023-04-1615:22andersmurphy(at least until something weird happens like cljs on both server and client)#2023-04-1615:22borkdudeyes, something like this.
I think how it should work:
The hook returns nil if the cljc language is clj in the e/client hook#2023-04-1615:23borkdudeand else it just expands into the original body#2023-04-1615:23borkdudeyou can't generate reader conditionals as those are processed already before you get to the hooks#2023-04-1615:23borkdudebut the cljc language is passed to the hook, so the e/client call is called for both clj and cljs#2023-04-1615:24andersmurphyNice I’ll give that a try (And let you know how it goes). Will it handle nesting? I.e e/client in an e/server in an e/client?#2023-04-1615:27borkdudeI think you'd need to handle that nesting yourself, not sure what the correct semantics are#2023-04-1615:28borkdudee.g. if you have e/client what does it mean if you also have an e/server block inside of that?#2023-04-1615:29andersmurphyjust switches the context. Everything in this block is on the server unless its in an e/client… etc. So should be just switching between cljs/cljc and clj/cljc?#2023-04-1615:30andersmurphyso as long as e/client and e/server themselves are treated as cljc it should be ok?#2023-04-1615:30borkdudeit would have been a lot easier of electric had chosen reader conditionals for this maybe? at least for clj-kondo ;)#2023-04-1615:31borkdudeif it works like you said then my idea to not return anything in e/client for the clj side wouldn't work since that can also contain e/server blocks?#2023-04-1615:31andersmurphycorrect#2023-04-1615:32borkdudeso maybe the clj side should just filter out the e/server calls and return those?#2023-04-1615:32andersmurphyI can’t speak for the electric team but I Imagine part of the challenge, is that the server doesn’t necessarily have to be clj in future#2023-04-1615:33borkdudeok, let's just try and see what it leads to#2023-04-1616:48xificurCThis is tricky. E.g. this is valid but will warn
(e/server
(let [x 1]
(e/client x)))#2023-04-1617:27andersmurphyI think that might be fine if things defined/bound in e/ function/macros are treated as cljc I.e valid for both client (cljs) and server (clj)?#2023-07-2414:25telekid> I can’t speak for the electric team but I Imagine part of the challenge, is that the server doesn’t necessarily have to be clj in future
Here's a weird proposal. What if kondo supported a special tag that indicated a change in dialect in the subsequent expression?
(e/client
#_{:clj-kondo/lang :cljs} ;; (or #_{:clj-kondo/lang #{:cljs}} for multiple targets)
(e/server
#_{:clj-kondo/lang :clj}
...))
This annotation would automatically make any rule that is sensitive to dialect work in languages like electric, where expressions can indicate changes in execution environment. It also decouples location (client / server) from dialect, solving the "server doesn't necessarily have to be in clj" problem.
~Wait, I've just re-invented reader conditionals. Where's my coffee?~
This is different from reader conditionals, because reader conditionals aren't designed to be nested. That is, AFAIK in #?(:cljs (e/client #?(:clj (e/server ...))) , everything inside #(:clj ...) is effectively a noop.#2023-07-2414:42Dustin Getzreader conditionals don't work in electric clojure unfortunately, because they expand at read time#2023-07-2414:51telekid> reader conditionals don't work in electric clojure unfortunately, because they expand at read time
That's subtle and interesting.#2023-04-1617:02Adrian AleixandreI could use some help understanding an issue I’ve encountered 🧵#2023-04-1617:02Adrian AleixandreI have an electric function that is intended to be a component that, by default renders a heading,
but has an edit button that transforms the heading into an input that you can then save and turn
back into a heading. (Imagine the behavior of the title field of a GitHub PR for example).
This is the sort of structure that I have now:
clojure
(e/defn EditableHeading [value OnChange]
(let [!editing? (atom false)
editing? (e/watch !editing?)
!temp-val (atom value)
temp-val (e/watch !temp-val)]
(dom/div
(dom/props {:class (dom/class-str ["flex" "gap-4" "justify-between" "items-center"])})
(if editing?
(dom/input
(dom/on "input" (e/fn [e]
(reset! !temp-val
(.. e -target -value))))
(dom/props {:value temp-val
:class (dom/class-str ["w-full" "border-b" "border-blue-500" "py-2" "px-2"])}))
(dom/h3 (dom/text value)))
(dom/button
(dom/on "click"
(e/fn [_]
(if editing?
(do
(OnChange. temp-val)
(reset! !editing? false))
(swap! !editing? not))))
(dom/text (if editing? "Save" "Edit"))))))
My issue I’m seeing is the following. Clicking “Edit” correctly puts the component into editing mode,
after which I can edit the input field. However, upon clicking save, the OnChange electric fn is called
as expected but the reset! that should put the component back into the normal non-editing mode
appears to not work. If I comment out the call to OnChange, the reset! does work.
What could be going on here?#2023-04-1617:02Adrian AleixandreHere’s a short video of the issue:#2023-04-1617:16Dustin Getzi’ll have to take a close look later, in the meantime you seen the TodoMVC demo in the main repo? it has approximately this functionality#2023-04-1617:17Adrian AleixandreI have not yet checked out that example — I’ll take a look#2023-04-1617:18Dustin Getzwhat does OnChange do? does it talk to the server?#2023-04-1617:18Adrian AleixandreYes it sure does#2023-04-1617:19Adrian AleixandreIn fact, if I remove the e/server from inside it the issue goes away — I think you might be onto something#2023-04-1617:21Dustin Getzthere is a cute little hack in the todomvc example where i think we use (case) to work around something like this - this is an area of active work right now and new controls and input primitives are coming this month#2023-04-1617:22Adrian AleixandreTotally understandable! I realize I’m riding on a ship that is still under construction, haha.
I’ll look for that example or just restructure my UI to dodge the problem.
Appreciate the help Dustin#2023-04-1617:23Dustin Getz(case (X.) (y!)) , if X throws pending, will delay y! until the pending is resolved. which kinda makes sense because until the X value is known we can’t test it to decide which conditional branch to run#2023-04-1617:25Adrian AleixandreBingo! I was looking for a way to force the operations to be sequential but didn’t think of using case#2023-04-1617:25Adrian AleixandreWhoops, didn’t mean to send that to the channel facepalm#2023-04-1617:26Dustin Getzyeah, i doubt this will be the final idiom but in the meantime this definitely happens and i think this is the best pattern we have #2023-04-1617:26Adrian AleixandreWorks for me! Maybe there needs to be an (e/do ...) or something of the sort#2023-04-1617:25Adrian AleixandreBingo! I was looking for a way to force the operations to be sequential but didn’t think of using case#2023-07-2414:25telekid> I can’t speak for the electric team but I Imagine part of the challenge, is that the server doesn’t necessarily have to be clj in future
Here's a weird proposal. What if kondo supported a special tag that indicated a change in dialect in the subsequent expression?
(e/client
#_{:clj-kondo/lang :cljs} ;; (or #_{:clj-kondo/lang #{:cljs}} for multiple targets)
(e/server
#_{:clj-kondo/lang :clj}
...))
This annotation would automatically make any rule that is sensitive to dialect work in languages like electric, where expressions can indicate changes in execution environment. It also decouples location (client / server) from dialect, solving the "server doesn't necessarily have to be in clj" problem.
~Wait, I've just re-invented reader conditionals. Where's my coffee?~
This is different from reader conditionals, because reader conditionals aren't designed to be nested. That is, AFAIK in #?(:cljs (e/client #?(:clj (e/server ...))) , everything inside #(:clj ...) is effectively a noop.#2023-04-1717:34Tim BrownHey folks... I'm having great fun with Electric, but have come across an issue and hope you can help me.
I have an atom, defined in a different namespace to where my electric code lives. e/watch seems not to react to changes to this atom.
Are there any rules that I should be aware of?
(ns app.tim (:require
[hyperfiddle.electric :as e]
#?(:clj [app.my-lib :as m])))
(e/def state (e/server (e/watch m/!state)))
(e/defn My-app []
(e/server (prn "update detected" state)))
(comment
(swap! m/!state update :foo inc) ;; bang on this... nothing happens :-(
)
(ns app.my-lib)
(defonce !state (atom {:foo 0))#2023-04-1717:35Dustin Getzhi, please post the full source code#2023-04-1717:35Dustin Getzthere are no special rules here#2023-04-1717:36Dustin Getz(ns app.tim (:require #?(:clj [app.my-lib :as m]))) I'm not sure if this passes the CLJS compiler#2023-04-1717:36Dustin Getzit expands to (ns app.tim (:require)) which fails spec validation for :require#2023-04-1717:38Tim BrownI don't believe that m/!state is required for the CLJS...#2023-04-1717:39Tim BrownOh... ISWYM... I have other libs being required here (which is why I get way with that)#2023-04-1717:40Tim BrownI've changed the above; but I haven't tested code this simple out.#2023-04-1717:41Tim BrownThe atom I have contains a very large collection of email headers#2023-04-1717:41Dustin GetzHere is a working example of watching an atom from another namespace: https://github.com/hyperfiddle/electric/blob/0bc5f4bcd96b632f4522de328af66b44f1220fe4/src-docs/user/demo_todomvc_composed.cljc#L24#2023-04-1717:42Tim Brown[I'll look at that]#2023-04-1812:00Dustin GetzSeems like Three.js should work with electric, by issuing point mutations to the scene graph with Electric and then having the entrypoint call renderer.render (with a forced causal dependency on animation time). Scene graph construction can be refactored into the declarative style using the same approach as electric-dom, by using macros to maintain a focal point for writes in dynamic scope. A few hours work at intermediate understanding, use electric-dom as a reference (it's very simple)#2023-04-1816:43denikany tricks to speed up compilation? have 4 sec+ recompile/reload cycles on a maxed out 2023 macbook pro#2023-04-1817:13xificurCCan you comment out namespaces you're not working on? Or change the entrypoint?#2023-04-1817:13deniksure it gets much faster then#2023-04-1817:14denikbut sometimes that’s not so easy like when I have components deeply nested in UI context#2023-04-1817:16xificurCYeah, sorry, no other tricks up my sleeves right now. We're working towards fixing compilation times but there are other tickets we need solving first#2023-04-1817:17denikokay#2023-04-1817:24Dustin GetzIf the recompiles are related to style changes, using an external CSS toolchain can reload css without triggering an Electric rebuild#2023-04-1817:55denikusing tailwind so a lot of styles are added as classes which triggers compiles#2023-04-1914:01arohnerI’m still struggling to build reusable components, because e/fn is not serializable, atom is not serializable and a/chan is not serializable. Are there any other tricks I can use to wire together reusable components?#2023-04-1914:08Dustin GetzShow me your source code, channels and atoms and other reference types can always be turned into reactive values and the values do move. e/fn not moving is indeed an annoyance, this is getting fixed soon, I will show you some tricks for dealing with it in the meantime but I need to understand exactly what you're trying to do#2023-04-1914:09arohner(e/defn Table [{:keys [columns rows row-format column-sort column-format]}]
(let [row-format (or row-format
(e/fn [row col]
(assert col)
(assert row)
(e/client (dom/text (col row)))))
column-format (or column-format
(e/fn [col]
(assert col)
(e/client (dom/text col))))]
(println "table")
(e/client
(dom/table
(dom/props {:role "grid"})
(dom/tbody
(dom/thead
(dom/tr
(e/for [col columns]
(dom/th (column-format. col)
(dom/on "click" (e/fn [e]
(e/server (println "click" col)
(reset! column-sort col))))))))
(e/for-by :db/id [row rows]
(dom/tr
(e/for [col columns]
(dom/td
(row-format. row col))))))))))#2023-04-1914:10Dustin Getzand the usage? I dont see any atoms or channels#2023-04-1914:11arohner(e/defn Artists []
(e/client
(dom/h1 (dom/text "Artists"))
(let [cols [:v]
!search (atom "")
search (m/watch !search)
{:keys [::search] :as s} history/route]
(ui/input search (e/fn [v]
(history/swap-route! assoc ::search v))
(dom/props {:placeholder "Search" :type "search"}))
(Table. {:columns cols
:rows (e/server
(->> (if (seq search)
(-> (fmb/artists-starting-with db search {:limit 100})
(mx/seq-consumer))
(fm/datoms> db :avet [:artist-credit/name] {:limit 100}))
(m/eduction (map (fn [d]
{:e (:e d)
:a (-> d :a :ident)
:v (:v d)})))
(m/reductions conj [])
(m/relieve {})
new))
:column-sort (atom nil)
:row-format (e/fn [{:keys [e a v]} col]
(e/client (history/link [::artist e] (dom/text v))))}))))#2023-04-1914:13arohnerthis is basically just a simplified gridsheet and musicbrainz. I’d like to make Artist and Table separate things, so that sort behavior is pluggable#2023-04-1914:13arohneronce I had :column-sort observable, I can update the query#2023-04-1914:15Dustin Getzthe column-sort atom is meant to be on the client right? it's being reset! from server#2023-04-1914:19Dustin GetzBeyond that I don't see any specific problems here, I will give you some tips in a moment. Specifically I dont see an issue related to e/fn not transfering in this example, am i missing it#2023-04-1914:50xificurC(m/watch !search) returns a missionary flow on the client which you try to use on the server. You probably meant (e/watch !search) which returns a reactive value#2023-04-1914:51Dustin Getzoh wow i missed that#2023-04-1914:54xificurCyou're passing a mutable reference (`column-sort`). Can you either create it inside Table or pass a pre-sorted dataset?#2023-04-1914:56arohneraha, m/watch was it. I didn’t assume that was a big deal because:
(doc e/watch)
-------------------------
hyperfiddle.electric/watch
([!x])
Macro
for tutorials (to delay teaching constructor syntax); m/watch is also idiomatic
#2023-04-1914:57arohnerI’m trying to build a reusable table with sorting and filtering. If I don’t pass mutable references how do I do that?#2023-04-1914:57Dustin Getzthanks ill fix the docstring, the equivalent to e/watch is (new (m/watch ...))#2023-04-1915:01arohnerthe dataset is too big to fit in ram or stream over the network, so I want the server side query to change in response to the column sort changing#2023-04-1915:06xificurCsince you were searching for unserializable reference transfers I was trying to reduce the number of moving parts. If the m/watch fixed it the atom is fine#2023-04-1915:08Dustin Getzhttps://gist.github.com/dustingetz/a6f4f8abd62eec4e2a5a4c91b2f9a3cd#2023-04-1915:09Dustin GetzTo be clear I dont think you hit them in the exact snippet you showed me, but likely you have hit them in other factorings#2023-04-1915:11arohneryeah, thanks#2023-04-1915:12arohneraha, I didn’t realize e/defn and e/fn behaved differently#2023-04-1915:12Dustin Getzyeah, obviously they should not#2023-04-1915:15arohnerwhat does start on the server mean?#2023-04-1915:16Dustin Getz(e/defn Table [{:keys [columns rows row-format column-sort column-format]}]
(e/server ...))
(e/server (Table. ...))
#2023-04-1915:17arohnerIn
(e/defn Foo []
(+ 1 1))#2023-04-1915:17arohnerwhat context is that in?#2023-04-1915:17Dustin Getzit inherits the context of the call site#2023-04-1915:17Dustin Getztoday the Electric entrypoint is in cljs, so by default at the top your program starts on the client#2023-04-1914:03arohnermy goal is a table of data where I can click on a column header and sort the table. I feel confident I can build that as a single e/defn, but I’d like to build a reusable table component that multiple pages can use, where I can pass in e.g. :column-on-click-handler#2023-04-1914:06arohnerI could probably use e/def, but that’s a little gross and prevents using multiple tables on the same page#2023-04-1916:06arohneris it possible to write a function/macro that determines which context you’re in?#2023-04-1917:26Dustin GetzThrough an implementation detail yes it is technically possible today with a macro, but we aren't ready to commit to it. Like you, I have also reached for this a few times. Can I see what problem you're trying to solve with this?#2023-04-1918:03arohnerI was experimenting with making essentially e/atom, implemented by creating an atom on both sides, and then derefing the right one in the right context#2023-04-1920:22vincentProbably don't manually coordinate 2 atoms#2023-04-1920:22vincentif you wish to preserve sanity 😅#2023-04-2009:37braai engineerphew finally getting some ROI on my Electrical engineering degree
wd guys, Electric is great 👏#2023-04-2015:41braai engineerFeels like Electric could benefit from XTDB’s xtdb.api/open-q. Any examples of how to leverage that via an iterator to start streaming results early?#2023-04-2016:05Dustin GetzThis will require missionary knowledge and is a bit advanced, if you collect all the links for us I will make a ticket and we can take a look. I don't think we've actually tried to do this yet#2023-04-2016:49denikfor an app like https://github.com/lumberdev/tesserae how much heap is too much? I keep having to increase heap size. it’s at 3gb now which seems excessive given that the default max is 1gb trying to figure out if I have a memory leak#2023-04-2017:47Dustin GetzDo you have a reason to suspect Electric or Missionary is the problem? That project uses a lot of tech, I'd start by hooking up an observability tool and figure out what the memory is being used for#2023-04-2017:48Dustin GetzNote that Electric and Missionary fit in a browser tab#2023-04-2017:49denikkinda, I have been using the same stack for over a year and did not have that issue#2023-04-2017:55Dustin GetzVisualVM can monitor heap, we can probably find evidence#2023-04-2018:19denik2023-04-20T18:16:35.384295160Z app[web.1]: #error {
2023-04-20T18:16:35.384375682Z app[web.1]: :cause Java heap space
2023-04-20T18:16:35.384381835Z app[web.1]: :via
2023-04-20T18:16:35.384398609Z app[web.1]: [{:type java.lang.OutOfMemoryError
2023-04-20T18:16:35.384403353Z app[web.1]: :message Java heap space}]
2023-04-20T18:16:35.384407205Z app[web.1]: :trace
2023-04-20T18:16:35.384414990Z app[web.1]: []}
2023-04-20T18:17:17.205331675Z app[web.1]: Exception in thread "missionary scheduler" Exception in thread "missionary blk-0" java.lang.OutOfMemoryError: Java heap space
2023-04-20T18:17:17.205646191Z app[web.1]: java.lang.OutOfMemoryError: Java heap space
2023-04-20T18:17:17.208733411Z app[web.1]: Exception in thread "logback-1" java.lang.OutOfMemoryError: Java heap space#2023-04-2018:20Dustin Getzeverything is in a missionary stack frame in your app, right?#2023-04-2018:20deniknot everything#2023-04-2018:21denikthis also runs dockerized on a server so I’d need to look into how visualvm could plug in#2023-04-2018:29Dustin Getzi’d expect to be able to repro the problematic patterns on your local dev env even if you don’t get the OOM#2023-04-2018:30Dustin Getzblk-0, this might be coming out of a m/via#2023-04-2019:17denikmost likely coming from this file https://github.com/lumberdev/tesserae/blob/a909c2341037f2ee38107233b9564ac3cd34df93/src/tesserae/eval.clj#L311#2023-04-2020:13denik#2023-04-2020:13denikevery few memory requirements jump by a few hundred megs until it runs out and crashes#2023-04-2020:31Dustin Getz(m/via m/blk (eval-fn cell))#2023-04-2020:31Dustin GetzWhat volume are you putting through this#2023-04-2020:31deniknothing crazy. maybe 20 invocations an hour#2023-04-2020:32denikI disabled all code in eval and memory still did not go down#2023-04-2020:32deniknot 100% sure it’s the culprit#2023-04-2020:37Dustin Getzwe know the OOM happened on a thread from the missionary blk threadpool
It's possible that it's a coincidence, but seems unlikely
It also seems likely that SCI allocates nontrivially to parse/interpret/run, it's a whole language with it's own allocations
So the question to me is, who is leaking?
It could be:
• sci itself
• userland usage of sci
• missionary itself
• userland usage of missionary#2023-04-2020:38Dustin GetzSCI's caching strategy could be interacting badly with the missionary idioms#2023-04-2020:39denikright or electric#2023-04-2020:39Dustin Getz"I disabled all code in eval and memory still did not go down" I dont understand yet#2023-04-2020:39denikdisabled eval right now so it’s to seen whether memory consumption grows#2023-04-2020:39denikif yes the issue lies outside all code inside the eval ns#2023-04-2020:39Dustin Getzyou mean you rebooted the app without sci evaluations and are watching the memory chart?#2023-04-2020:40denikhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1682023142039899?thread_ts=1682009368.316359&cid=C7Q9GSHFV
(mount/stopped) it in an nrepl#2023-04-2020:40Dustin Getzeval-schedule-listener ?#2023-04-2020:41denikall of the defstates in that ns#2023-04-2020:41Dustin GetzThere are only two (m/via m/blk ...) in this file, are they both in eval-schedule-listener?#2023-04-2020:41Dustin Getzare there any other (m/via m/blk)s in your app?#2023-04-2020:41deniknope#2023-04-2020:42Dustin Getzok, so now we wait and see#2023-04-2020:44Dustin GetzThe missionary userland code could be retaining references to what ever structure SCI returns, including the results#2023-04-2020:44denikthere’s also a possiblity that some code sci is running that’s using clojure.core.memoize/ttl is the issue#2023-04-2020:44Dustin Getzdoes your app contain m/signal! or m/stream!#2023-04-2020:45denikQ is would data allocated through sci and mem/ttl land in the heap?#2023-04-2020:45denik@U04V15CAJ?#2023-04-2020:45Dustin Getzmy knowledge of java memory model is primitive, i thought mostly everything was on the heap#2023-04-2020:46denikhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1682023497728309?thread_ts=1682009368.316359&cid=C7Q9GSHFV no, not as usercode#2023-04-2020:46denikthe max value of the heap seems to small for everything to be in it#2023-04-2020:46Dustin GetzIIUC, the only missionary operators that allocate are m/signal! and m/stream!, and Electric Clojure let forms compile to m/signal!#2023-04-2020:47Dustin Getzso if something downstream of the sci evaluation is being retained, it would be the Electric code is retaining the most recent previous evaluation#2023-04-2020:47Dustin Getzbut not like, the history of evaluations#2023-04-2020:48borkdudeI was pinged in this thread. Can you please summarize your question so I don't have to read the whole thread?#2023-04-2020:48Dustin Getzit would make more sense that a sci result had deep references into caches#2023-04-2020:49denik@U04V15CAJ is there possibility that code using clojure.core.memoize/ttl that is being evaluated through sci causes a leak in heap memory?#2023-04-2020:50borkdudewhy would you think that would happen?#2023-04-2020:51denikI seem to have a heap memory leak of code being executed by sci that uses clojure.core.memoize/ttl 😄#2023-04-2020:51denikbut there could be other reasons#2023-04-2020:51borkdudeyou'll have to be more specific#2023-04-2020:52borkdudeI have had some memory leaks in clj-kondo once where I captured the SCI context using memoize. So it could be a similar issue, but only a repro and more info will tell#2023-04-2020:52denikthis is a whole can of worms. will need to investigate more to provide more detail#2023-04-2020:52borkdudeit depends on what you are memoizing#2023-04-2020:53borkdudemy solution in clj-kondo was to memoize only data, not the SCI context itself#2023-04-2020:53denikhttps://github.com/lumberdev/tesserae/blob/72735350890fa18dfb1713c224e28cc1fe40c9fa/src/tesserae/eval.clj#L65#2023-04-2020:53denikmemoizing http requests#2023-04-2020:53Dustin GetzHow can we measure the size of that sci-ctx reference#2023-04-2020:54borkdude@U050CJFRU This was the fix I made in clj-kondo: https://github.com/clj-kondo/clj-kondo/blob/a284300c3ed0c0c201d3d4933d22323ed44bd2d1/src/clj_kondo/impl/hooks.clj#L84-L92
(but again, not sure if this is your issue)#2023-04-2020:56denikthanks. that does seem different#2023-04-2020:56borkdudeI see a link to a creation of a SCI context and then I hear that you are memoizing http requests, not sure what the connection is#2023-04-2020:56denik@U09K620SG just repro’d on dev#2023-04-2020:56denikException in thread "clojure.core.async.timers/timeout-daemon" java.lang.OutOfMemoryError: Java heap space
Thu Apr 20 16:54:16 EDT 2023 [client-loop] ERROR - select exception, should not happen
java.lang.OutOfMemoryError: Java heap space
Thu Apr 20 16:54:28 EDT 2023 [client-loop] ERROR - select exception, should not happen
java.lang.OutOfMemoryError: Java heap space
#2023-04-2020:57denikhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1682024188639159?thread_ts=1682009368.316359&cid=C7Q9GSHFV
@U04V15CAJ the only connection is that the code that does the ttl-memoization runs inside the sci context#2023-04-2020:57Dustin GetzThat is from datalevin i guess?#2023-04-2020:59deniknot sure. what would help from visual vm?#2023-04-2020:59denikheap dump?#2023-04-2020:59borkdudeok, other than the thing I pointed to earlier, this doesn't ring any bells. the sci context contains a lot of stuff but if you have only one context and are not creating new ones over and over (like I did in clj-kondo) I don't see how it can lead to a memory leak right now#2023-04-2020:59borkdudeso feel free to ping me again when you know more :)#2023-04-2020:59denikthank you @U04V15CAJ much appreciated!#2023-04-2021:00Dustin GetzI'm not the java expert on my team, i really only know how to sniff around#2023-04-2021:00Dustin Getzwhere is the d/q call in your codebase#2023-04-2021:02denikit’s using entity throughout#2023-04-2021:02Dustin GetzI'm not seeing evidence of a Missionary leak, Missionary is brutally low level and the state allocations are very explicit in the primitives, i think it is something in userland (that can be interacting badly with reactive primitives surely)#2023-04-2021:02denikall of that is in tesserae.db#2023-04-2021:02denikyes could be my fault. just trying to find the issue#2023-04-2021:02deniknever had this happen before which is why I posted here#2023-04-2021:03Dustin Getzwho calls d/entity, the UI?#2023-04-2021:03Dustin Getzsheet..cljc#2023-04-2021:04denikyes and code in eval#2023-04-2021:04Dustin Getzhow big is your db, tiny rigjht?#2023-04-2021:04Dustin Getzother than the code strings stored in it#2023-04-2021:04denikyup#2023-04-2021:05Dustin GetzI think if you restart the app fresh, with sci disabled, we can watch in visualvm for which classes bubble up to the top of the allocations#2023-04-2021:18denikjust did#2023-04-2021:34denik@U053XQP4S heapdump below in case you’re interested. from what I can tell the leak is occuring solely between electric & datalevin.#2023-04-2021:35denikheap slowly kept growing through many GC cycles. I only interacted with the UI.#2023-04-2021:48denikthere definitely seems to be a leak because when the whole system gets shut down (running mount/stop), a manual GC run, the heap usage remains unchanged#2023-04-2022:00denik#2023-04-2022:00denik#2023-04-2022:01denikthese TCP connection threads allocate tons of data#2023-04-2022:16Dustin Getz(n mobile) electric is serializing data en route to the client with transit, it’s okay to allocate as long as it’s not retained, and our tutorial app has heavier network than the spreadsheet does right? our prod metrics don’t show signs of a leak#2023-04-2022:16Dustin Getzin the network layer *#2023-04-2022:17Dustin Getzwhat is datalevin architecture?#2023-04-2105:08denikBtw unfortunately don’t have the stack trace available but my suspicion is that the core.async error comes from shadow-cljs. https://clojurians.slack.com/archives/C7Q9GSHFV/p1682024201136609?thread_ts=1682009368.316359&channel=C7Q9GSHFV&message_ts=1682024201.136609#2023-04-2114:17Dustin Getzin prod shadow should not be loaded at runtime, it is only used in dev for hot code reloading#2023-04-2114:45denikThis was in dev#2023-04-2114:45Dustin Getzyes but the OOM occurs in prod as well#2023-04-2114:46denikYes wanted to make clear that they are unrelated.#2023-04-2114:47Dustin GetzI dont understand#2023-04-2115:14Dustin GetzOk, the huge byte arrays are code strings from the shadow compiler:#2023-04-2115:17Dustin Getzshadow adds a huge amount of noise to the heap dump, I"d be interested in seeing a heap dump from localhost but using the prod build#2023-04-2118:11denikhere you go#2023-04-2118:12denikheap kept climbing slowly. increase of about 10mb/hour that GC would not be able to free up. Interestingly, interaction with the app did not seem to affect heap size after GC but time did#2023-04-2118:25Dustin Getzgood clue, i’ll think about that#2023-04-2118:26Dustin Getzuserland is running scheduled jobs from missionary discrete flows and m/sleep?#2023-04-2118:29denikactually false alarm. heap just dropped to previous level around 130mb. Not sure what happened in between#2023-04-2118:46denikwhat I can say for sure is that memory leaks after db transacts#2023-04-2118:50denikhere’s a heapdump where a few db transacts lead to about 60 mb increase in heap that did not clear up after GC. now investigating whether this is datalevin or electric/missionary#2023-04-2118:53denikthe above is with electric running#2023-04-2118:54denikjust confirmed making the same transactions without electric brings heap back to a baseline after GC#2023-04-2118:55denikhere’s the heapdump for that#2023-04-2118:55denikso it does seem that some internals of electric/missionary combined with datalevin ultimately leak memory.#2023-04-2118:56denikFYI @U053XQP4S#2023-04-2119:23Dustin Getzi don’t think we can draw any strong conclusions yet, now that we’ve isolated the issue to the transact i will look again at the app (likely not until tuesday)#2023-04-2119:24Dustin Getztransact returns something large (references to prev database etc), retaining that pointer could be retaining the entire chain of dbvals#2023-04-2119:25Dustin Getztry wrapping transact with clojure to return nil (thus guaranteeing that electric never touches that result)#2023-04-2119:26denikthe way it’s implemented (if you look at the eval ns) transacts that cause the leak happen outside of electric#2023-04-2119:26denikbut electric keeps a watches the db & transacted entities and that seems to have sth to do with the leak#2023-04-2119:28Dustin Getzhm ok on mobile will take a look next week#2023-04-2417:36denik#2023-04-2417:37denikran this simple test against datalevin while watching heap. Unlike the combination with electric, heap returns to previous levels after GC#2023-04-2417:45Dustin Getzcan you repro the leak in a unit test with electric?#2023-04-2417:46deniknot sure how I would do that#2023-04-2417:46deniksounds like it would need telemetry of the JVM#2023-04-2417:47denikI’m trying to move forward building my app. currently seeing if an implementation using d/listen! avoids the leak#2023-04-2417:48Dustin Getzif it is as simple as electric directly at fault, then we can take that minimal test you wrote and add one line to make it an electric test and see the problem#2023-04-2417:48denikwhat is the test case?#2023-04-2417:48denikultimately it leads to an out of memory error#2023-04-2417:48Dustin Getzscreenshot the heap#2023-04-2417:49denikhow can I put that into an electric test 😄#2023-04-2417:49Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_1_lang.cljc#2023-04-2417:50Dustin Getzshow me a 12 LOC program using datalevin and electric that leaks#2023-04-2417:50Dustin Getzdoesn't matter if it is a unit test or not#2023-04-2417:50denikah ok#2023-04-2417:59denik#2023-04-2417:59denik@U09K620SG will this do?#2023-04-2418:02Dustin Getzsure, as minimal as you can make will help us get to the bottom of this faster#2023-04-2418:02Dustin Getzplease add the heap dump and if you have a repo we can clone to easily reproduce the issue that would help#2023-04-2418:35denik#2023-04-2418:51denikalso checking with the author of datalevin to see if it could have sth to do with it#2023-04-2418:51denikgenerally observing heap growth with datalevin alone as well though at a much slower rate#2023-04-2418:57Dustin Getzwhat is the exact source code that matches the heap dump?#2023-04-2418:59denikin this case it’s cherrypicked parts of the tesserae repo.#2023-04-2622:49denikstill testing but it appears the https://github.com/juji-io/datalevin/issues/204 could have been in datalevin and was amplified by electric. currently testing with datalevin master and heap memory seems to consistently return to initial levels.#2023-04-2622:50denikfyi @U0A74MRCJ#2023-04-2622:54HuahaiRight, this is fixed in datalevin master branch.#2023-04-2700:49Dustin Getzis electric slamming the transaction in a loop or something like that? how frequently was electric invoking the leaky operation?#2023-04-2716:11denikissue is still there 😕#2023-04-2716:14HuahaiWhat are the objects with large numbers?#2023-04-2716:15denikmy bad, I think shadow-cljs bloated the heap#2023-04-2716:15denikturned it off, and usage went back to normal levels#2023-04-2108:32braai engineerHow can I tell Hyperfiddle how to serialize things like Dates so I don’t get this INFO log? I have time-literals installed on front & back, so it should know how to serialize #time/date "2018-10-09" (just like that).
: Unserializable reference transfer: java.time.LocalDate 2018-10-09
#2023-04-2109:09xificurCdata is transferred with transit, so you need to install the transit handlers in the app. Here's a https://github.com/hyperfiddle/electric/blob/0bc5f4bcd96b632f4522de328af66b44f1220fe4/src-docs/wip/demo_custom_types.cljcshowing how you can do that today#2023-04-2112:36braai engineerhmm, the following does not seem to work:
(ns xtx.transit
"Connect time-literals to transit.
Sourced and edited from:
"
(:require
[time-literals.read-write]
[cognitect.transit :as transit]
#?(:cljs [java.time :refer [Period
LocalDate
LocalDateTime
ZonedDateTime
Instant
ZoneId
DayOfWeek
LocalTime
Month
Duration
Year
YearMonth]]))
#?(:clj (:import
(java.time Period
LocalDate
LocalDateTime
ZonedDateTime
Instant
ZoneId
DayOfWeek
LocalTime
Month
Duration
Year
YearMonth))))
;; uhmm I think these have to be fully namespaces
(def time-classes
{'period Period
'time/date LocalDate
'date-time LocalDateTime
'zoned-date-time ZonedDateTime
'instant Instant
'time LocalTime
'duration Duration
'year Year
'year-month YearMonth
'zone ZoneId
'day-of-week DayOfWeek
'month Month})
(def write-handlers
(into {}
(for [[tick-class host-class] time-classes]
[host-class (transit/write-handler (constantly (name tick-class)) str)])))
(def read-handlers
(into {} (for [[sym fun] time-literals.read-write/tags]
[(name sym) (transit/read-handler fun)])))
;; Demo shows how to serialize custom types in Electric
;; Todo cleanup, there are better ways to do this
#?(:clj (alter-var-root #'
merge write-handlers)) ; server: write only
#?(:cljs (set!
(merge read-handlers))) ; client: read only
maybe because tags not fully-namespaced? cc @U09FL65DK#2023-04-2113:02xificurCI'd suggest taking a single class, e.g. Period, use it manually and get that working. Then you can expand to subsume above.
Any chance you're failing to transfer a date/time type from client to server? The config only sets up server->client#2023-04-2113:03xificurC#2023-04-2113:08braai engineerHmm, even this complains if I try to send a LocalDate (from server to client only):
(def write-handler (transit/write-handler
(fn [_] 'time/date) ; this tag must be namespaced!
str))
(def read-handler (transit/read-handler 12345.67)) ;; just overriding it for now to see if it can send from server to client
#?(:clj (alter-var-root #'
assoc LocalDate write-handler)) ; server: write only
#?(:cljs (set!
(assoc
"time/date" read-handler))) ; client: read only
Do I need to call alter-var-root from somewhere specific? Error:
INFO : Unserializable reference transfer: java.time.LocalDate 2000-01-01
Tried with string tag vs symbol tag.#2023-04-2113:32xificurCHere's a minimal working example where a JVM LocalDate is represented as a string "LOCALDATE: <date-here>" on the client#2023-04-2114:12Dustin Getzlets change the tutorial to this#2023-04-2114:13Dustin Getzalso tutorial should provide both directions imo#2023-04-2114:14Dustin Getzand also link to any supporting documentation/examples for transit#2023-04-2117:19braai engineerI have to re-eval the transit handler namespace on server to get changes to show up, which I find unexpected given that my “app” namespace :require’s my transit handler namespace.
The auto-builder does not seem to pick up changes to that namespace, which is confusing when trying to debug why handlers aren’t working.#2023-04-2117:50braai engineerHmm, maybe because alter-var-root is in #?(:clj …) directive?#2023-04-2109:27braai engineerHmm, can’t seem to use Specter in Electric functions, which kind of makes sense with all it’s crazy macros:
(->> (:entry/_tx tx)
(S/transform [S/ALL :entry/amount] float)) ;; because (sort-by :entry/amount ...) chokes on bigdec [TaggedValue f...] some or other.
Unable to resolve symbol: pathcache83102
{:file "xtx/electric.cljc", :cljs.analyzer/no-resolve true, :no-doc true, :private true, :in [pathcache83102]}
ExceptionInfo: Unable to resolve symbol: pathcache83102
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:643)
hyperfiddle.electric.impl.compiler/analyze-form #2023-04-2109:31leonoelI suspect the problem to be specter macroexpanding to inline defs, which in turn confuse electric compiler because the symbol doesn't refer to a top-level var#2023-04-2109:33xificurC• [TaggedValue f...] seems like a type you didn't expect, maybe the serialization returns something else than you expect?
• I think specter will work if you call it in a clojure context, not electric. As a first step try to push the specter code in a toplevel cc/defn#2023-04-2111:29Dustin Getzwe have a bug related to certain things that should be float coming across as TaggedValue ratio#2023-05-1110:15braai engineerIs there a quickfix for TaggedValue? I’m seeing decimals come over the wire as e.g. [TaggedValue: f, -2402.97] or should I coerce to float or string before sending from server to client?#2023-04-2119:00vincentI have a crazy idea- I'm gonna use my cloud box to do the heavy lifting for me, and code via SSH 😄 probably lots of fellows already doing this i bet#2023-04-2119:25vincentI think I found an issue w/ electric-examples-app a fresh clone gives. me
cloning:
Checking out: at 447810b9401d66d6f6d5a7bf3da685f406efa7a5
Cloning:
Downloading: com/google/guava/guava/31.1-jre/guava-31.1-jre.pom from central
Downloading: com/google/guava/guava-parent/31.1-jre/guava-parent-31.1-jre.pom from central
Downloading: thheller/shadow-cljs/2.22.10/shadow-cljs-2.22.10.pom from clojars
Downloading: markdown-clj/markdown-clj/1.11.4/markdown-clj-1.11.4.pom from clojars
Checking out: at ac038ebf6e5da09dd2b8a31609e9ff4a65e36852
Error building classpath. Failed to read artifact descriptor for com.datomic:dev-local:jar:1.0.243
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.datomic:dev-local:jar:1.0.243
... 21 more
#2023-04-2119:25vincentwhich is `Could not transfer artifact com.datomic:dev-local:pom:1.0.243 from/to cognitect-dev-tools (): status code: 401, reason phrase: Unauthorized (401)
#2023-04-2119:27Dustin Getzdatomic dev-local requires registration and out of band maven configuration, you can remove the datomic from deps.edn and comment out any references to the datomic example #2023-04-2119:33vincentI replaced it :deps {com.datomic/datomic-free {:mvn/version "0.9.5697"}
in deps.edn and seems to be working#2023-04-2119:34vincentNow I get The required JS dependency "@codemirror/language" is not available, it was required by "contrib/electric_codemirror.cljc".
maybe npm dep?#2023-04-2119:35vincentI do want to use datomic, just not sure how to accommodate the codemirror thang, but i'm looking into it 😅#2023-04-2119:36Dustin Getzyarn#2023-04-2119:37Dustin Getzor, npm install#2023-04-2119:37Dustin Getzdatomic free is not a drop in replacement i don’t think#2023-04-2119:41vincentconfirmed it is not 😅#2023-04-2119:42vincentSo ideally there would be a super simple app with:z#2023-04-2119:42vincentdatomic free,
jetty server w/ session,
and electric chat app#2023-04-2119:43vincenttrying to carve a useful starting block of marble out of the example app 😅#2023-04-2119:44vincentMaybe I can just add datomic to the starter-app and be good to go, I just require sessions, eventually, and figure it better to work with them from the outset#2023-04-2119:47Dustin Getzyes i recommend that, use the starter todos app and port datascript to datomic free#2023-04-2119:50vincentthanks for the rec#2023-04-2119:50vincentadd sesh later?#2023-04-2211:45Daniel Jomphethis little story above is a great example of how it might be great for Datomic to loosen up on the maven config requirements for dev-local tutorial-level users and tutorial-level creators (if possible)! @U1QJACBUM#2023-04-2223:06vincentuh yeah i don't want to touch maven with a ten foot pole 😅#2023-04-2223:06vincentwhat year is it 😅 😅 robin williams full of beard#2023-04-2121:24vincentI think my remote box ran out of JVM RAM 😅
I set the stack size to 3g is that too much ? o_O
2g was not enough#2023-04-2121:29vincent2777mb seems to do it x)#2023-04-2208:19braai engineerHas anyone done any work on deploying the https://github.com/hyperfiddle/electric-xtdb-starter to http://Fly.io? Right now, doesn’t have to scale to multiple regions, but would be nice if one region was the “master” node that others connected to and sync’ed up somehow. I just need to deploy my tax tool with persistent DB so my tax accountants can help categorize/flag expenses… and so it’s no longer tied to my machine.#2023-04-2212:04refset(replied here: https://clojurians.slack.com/archives/CG3AM2F7V/p1682165018638009?thread_ts=1677430221.688989&cid=CG3AM2F7V)#2023-04-2210:53braai engineerReactor complains about URL fragment in websocket URL on page reload e.g. localhost:8080/#home or new seems to inject a view in the wrong place or order?
(e/defn Link [!route route-key title]
(let [current-route (e/watch !route)]
(dom/a
(dom/props {
;:href (str "#" route-key)
:class (dom/class-str ["link" (if (= route-key current-route) "active")])})
(dom/on "click" (e/fn [_] (reset! !route route-key)))
(dom/text title))))
(e/defn Nav [!route]
(dom/div
;(dom/text "current:route:" (e/watch !route))
(Link. !route :home "Transactions")
(Link. !route :404 "Somewhere Else")))
(e/defn DefaultView [!route]
(e/client
(let [route (e/watch !route)]
(dom/div
(dom/text "missing page for route: " (pr-str route))))))
(e/defn MyApp [!route]
(e/client
(let [!route (atom :home)
route (e/watch !route)
;; changing !route is slow, so I thought let me cache the "views" (or signals):
tx-list (TxList.)
default-view (DefaultView. !route)]
; Hmm, Unexpectedly things are appearing in the wrong order...
(dom/div
(dom/h1
(dom/text "Test")
(dom/small
(Nav. !route)))
(case route
:home tx-list
(DefaultView. !route))))))#2023-04-2218:10braai engineerBtw. would be cool if Electric could add some meta-data to each dom node to show how long it took to render/delete that can be used to color the element background so I can see which nodes in the AST are slow, esp. if they trigger network traffic. Sort of like, “time spent in calculation” to spot hot nodes. cc @U09K620SG#2023-04-2220:16denike/dom renders by side effect which is why TxList and DefaultView render into the current e/dom node at evaluation time in the let#2023-04-2221:18braai engineerCan it render where I place it? Is there a way to defer the render until I place it somewhere?
The reason I want to “cache” the view is because it takes ~2 seconds to change from the TxList (a complex view) to the simple DefaultView. I tested it and it changes quickly between two simple views but takes a long time from the complex view, presumably because it’s doing a bunch of teardown - not sure how to profile it.#2023-04-2300:56Dustin Getzour tear downs are slow with large amounts of dom , this is a known issue that we haven’t looked into yet#2023-04-2300:57Dustin Getzyou can defer rendering by wrapping it in e/fn; dom/text has the same evaluation rules as println basically #2023-04-2223:07vincentoffhand somebody know the default web socket port?#2023-04-2223:14vincentbig fan so far
just need to get my dev box cooperatin'
then i'll be flyin'! 😄#2023-04-2223:42vincentOkay I had a question about getting the websocket working
#2023-04-2300:54Dustin Getzhey i’m out of town this weekend but i would direct you to the starter app repo which should contain a working app. that env var is checked in prod configurations to auto-refresh the page after a rolling deploy#2023-04-2300:55Dustin Getzyou can grep the repo (search on github should work) to find that env var and see the exact rules#2023-04-2302:45vincentThank you, and no rush dude, I appreciate any and all help, whenever is convenient for you.#2023-04-2302:45vincentI will check the repo more.#2023-04-2403:05vincentDude I am such a goon. There's the line
`
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty
`
in the flippin docs
I didn't notice that addition.
But,I still get the error in the web console, i think because not forwarding web socket port?
am invoking with
clj -A:dev -X user/main
#2023-04-2223:42vincentO_o#2023-04-2403:05vincentDude I am such a goon. There's the line
`
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty
`
in the flippin docs
I didn't notice that addition.
But,I still get the error in the web console, i think because not forwarding web socket port?
am invoking with
clj -A:dev -X user/main
#2023-04-2321:28vincentI'm thinking it will be easier with nginx vs apache so I am gonna transition some stuff n then get back to electric'ing x) ⚡#2023-04-2321:42vincentwow nginx is great 😅 that was painless lol. hope you're all having an amazing weekend. ☀️#2023-04-2322:04vincentserver {
listen 80;
server_name *.com;
location / {
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass ;
}
location /ws/ {
proxy_pass ;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
}
}
should I set the websocket port to something other than 80#2023-04-2507:26soulawakerHello! I’m looking the electric-starter-app. I have one question.
The https://github.com/hyperfiddle/electric-starter-app/blob/main/src/electric_server_java11_jetty10.clj requires “ring.middleware.cookies”.
But in the https://github.com/hyperfiddle/electric-starter-app/blob/main/deps.edn, it seems that no “ring.middleware” related deps like “ring/ring-core” are declared. How does this code get “ring.middleware” related deps?#2023-04-2507:41xificurCthe middleware namespaces are part of ring.core, which comes in as a transitive dependency from info.sunng/ring-jetty9-adapter#2023-04-2508:21soulawaker@U09FL65DK Thank you for reply!
I’m confused.
I’ve been thinking that directly using transitive dependencies of declared dependencies is not allowed in clojure.
Is it wrong?#2023-04-2509:32xificurCyou can require anything that is on the classpath#2023-04-2511:09soulawaker@U09FL65DK Thank you!#2023-04-2514:55Hans MoserHi all! I love playing with Electric, this is really cool stuff but I ran into some problems with form elements while modifying the starter app.
1. Is there an easy way to access the value (and possibly name) of a radio input? I naively tried the following, but the whole click event is passed to (e/fn[v]...).
(dom/form (dom/text "1%")
(dom/input (dom/props {:type "radio" :name "r1" :value "1"})
(dom/on "click" (e/fn [v] (reset! !ss (x->y v)))))
(dom/text "5%")
(dom/input (dom/props {:type "radio" :name "r1" :value "5"})
(dom/on "click" (e/fn [v] (reset! !ss (x->y v))))))
2. Using the same idea with checkboxes, they pass true or false to (e/fn) which usually makes sense. I'm wondering if there is a way to change the values though, e.g. through the props (dom/props {:id "cb1", :value "1"}) which doesn't seem to work.#2023-04-2516:04Dustin Getzcontrols in the dom namespace (i.e. dom/input) pass dom events to the callback for max low level dom power#2023-04-2516:04Dustin Getzcontrols in the ui namespace (i.e. ui/checkbox) get the value, these controls are intended to be higher level#2023-04-2516:06Dustin GetzWe haven't provided ui/radio yet,#2023-04-2516:09Dustin GetzI believe we are questioning if ui/radio is really just a styled picklist, similarly sets of checkboxes may also be picklists (as opposed to a single bool field on a record)#2023-04-2516:09Dustin Getzpicklists are pretty complicated, for example the khan academy course picker is showing a single page of results with overflow ("see all" button) so there is in fact server pagination happening on this simple checkbox picker, should a radio picker also have this functionality? considering a radio picker is just the single-select version of a checkbox picker which is multi-select?#2023-04-2516:11Dustin GetzWe have strong opinions on all of this and have built all of these widgets before, we just haven't yet gotten around to redoing them in Electric#2023-04-2516:12Dustin GetzIn the meantime you could implement your own radio control using the dom namespace if you want a higher level interface than the event#2023-04-2516:12Dustin GetzWe also have comprehensive crud form functionality which has not yet been revealed/documented, it supports optimistic updates and handles robust server sync, this is under active development as electric-ui5, hope to release the next version of this in May#2023-04-2519:06Hans MoserThanks a lot for your detailed reply, Dustin. I will give it a shot but it is great to know that an official form implementation is on its way. I still have a hard time wrapping my head around how it all ties together at this point.#2023-04-2612:02Dustin GetzHeaded to Clojure Conj, who else is going?#2023-04-2616:51Rob BI will be a virtual attendant and am looking forward to it.#2023-04-2617:32zaneI’ll be there!#2023-04-2620:11chromalchemyIm going#2023-04-2621:11alexdavisMyself and some of the Juxt/XTDB team will be there! Already had a few conversations with people here about electric #2023-04-2621:05markaddlemanI’m trying to incorporate plotly into a sample electric app. I’m trying to set up plotly click events to drive electric functions so I can easily call the server, change state, etc.#2023-04-2621:06markaddlemanWhen the “plotly_click” e/fn handler is called on line 15 (new V1 …) , I get an invalid arity exception#2023-04-2621:06markaddlemanWhat am I doing wrong?#2023-04-2621:12markaddlemanI have a feeling that the proper way to do this is that the plotly callback should mutate some atom and then the electric fn should watch that atom#2023-04-2621:35noonianI think you can only call electric functions from within an electric function body or an e/def, and the problem is that plot! Is a regular cljs function and in it you are trying to call V!#2023-04-2621:47xificurCYes, you should call dom/on with an e/fn#2023-04-2621:47markaddlemanWhat’s the story around interop? Plotly needs a plain javascript fn to handle its callback#2023-04-2621:49markaddlemanoh, maybe i get it… instead of (.on div ….) use dom/on ?#2023-04-2622:06markaddlemanI’ve updated the code to try dom/on . Now, I get a compiler warning Cannot infer target type in expression (. inst__30182__auto__ newPlot G__46562 G__46563) and it doesn’t seem that the callback is getting invoked at all.#2023-04-2623:19markaddlemanHere’s a working version. The big realization for me is that new is a special form (hence all the electric macros). This approach relies on plain Clojurescript for the interop and state atoms to get reactivity with electric.#2023-04-2702:40noonianAdding a ^js type hint to js/Plotly where you call .newPlot may help with the warning. That shouldn't affect behavior in development but will probably break things when you try to compile with advanced optimizations#2023-04-2711:34markaddlemanThanks!#2023-04-2712:57xificurCwe have a convention to name electric functions capitalized. Electric functions are called with new . Therefore if you want to run electric code on a DOM event you call
(dom/on div "plotly_click" (e/fn [data] ... (new V! ...)))
dom/on will call new on your e/fn
If a clojure function suffices you can use dom/on!
(dom/on! div "plotly_click" (fn [data] ...))
#2023-04-2710:31Lidor CohenHi all, I'm experimenting with reverse of control style: https://cycle.js.org/ i.e the computed state variable reads events from input and compute its state, (as opposed to the event handlers sets some state variable).
What is the most idiomatic way to extract an event stream from event handlers?
I know missionary can expose a very similar abstraction to Rx so I'm guessing that there should be some simple primitive I could use to get an event stream out of event handler.
Thanks in advance 😄#2023-04-2710:39Dustin Getzm/observe#2023-04-2711:08Lidor CohenIf I'm setting listeners on all events but no one is reading them are they still costing performance? or is there some kind of optimization on the graph that eliminates orphan signals?#2023-04-2712:12Dustin Getzevent callbacks are eager (due to how the platform works), use (m/relieve rf) to relieve backpressure by dumping events into an accumulator via reducing fn. the accumulator is then lazily sampled thereafter#2023-04-2712:13Dustin Getzsee http://electric.hyperfiddle.net section about backpressure and lazy clocks, and please ask clarifying questions here so i can improve the docs#2023-04-2712:19Lidor Cohen> • The clocks pause when the browser page is not visible (i.e. you switch tabs), confirm it
Wasn't able to confirm it, switched tabs but clocks kept on running#2023-04-2712:34Lidor Cohen@U09K620SG did you happen to think about the nature of signals vs imperative state (swap!) at the event handlers (either dom events or async tasks) boundaries?
This is where FPR seems to fall back to imperative, and I wonder about that:
• is this the best abstraction for "importing" data into the signals graph? if so why?
• what happens if we try to take the declarative (frp) approach all the way (like cycle.js did)?
If you had the chance to think about the nature of external inputs and how they fit best into the frp system I'd love to get some insight 🙂
Thanks#2023-04-2712:52xificurCelectric compiles to https://github.com/leonoel/missionary which supports discrete streams and continuous signals under a single protocol#2023-04-2717:05Dustin Getzm/observe is the way to import events into the dataflow graph#2023-04-2817:20vincentDatomic is free now...#2023-04-2819:55teodorluPerhaps we might hope for a Datomic in-memory example on https://electric-examples-app.fly.dev/?#2023-04-2821:46Dustin Getzyes we can do that now, the only reason to exclude it was because of the private maven dep#2023-04-2817:31denikwhat’s the best way in electric to compare a previous param to a e/fn to a current one?#2023-04-2817:32denikI’d like to add a css class when a certain value changed and remove it otherwise#2023-04-2818:10vincentMaybe clientside atom? I wonder if there is a nice binding form but it sounds like clientside#2023-04-2818:10denikactually realized this is even more complicated#2023-04-2818:11vincentClojure has atom observer stuff, no? like the default fn to run when the atom change#2023-04-2818:11denikclass needs to be removed after a certain time period because it’s an animation and won’t run consecutively unless fully removed and re-added#2023-04-2818:11vincentoh, that's effed#2023-04-2818:11vincentand mymistake it's just validation on input of atom#2023-04-2818:12vincentYou need some sort of time line coordination#2023-04-2818:12vincentto save sanity longterm imho 😅#2023-04-2818:14vincentor lots of callbacks?#2023-04-2818:16deniknot sure. looking into it#2023-04-2818:20vincentso this is because CSS animations need the component to be added and removed to get them to re-run, yes? You can define your own key-frames if you wanted to make an extra long animation.#2023-04-2819:37xificurC(defmacro after [ms form]
`(case (new (e/task->cp (m/sleep ~ms))) ~form))
#2023-04-2819:37xificurCThis allows running an electric form after given ms#2023-04-2819:41xificurCAnother one I sometimes play with
(defmacro keep-for [ms form]
`(case (new (e/task->cp (m/sleep ~ms) ::wait)) ::wait ~form nil))
#2023-04-2819:43xificurCEven full remove and readd might not trigger the css animation if done too quickly btw#2023-04-2819:47xificurChttps://css-tricks.com/restart-css-animation/ are some ideas I tried, I think (set! (.-offsetWidth dom/node) (.-offsetWidth dom/node)) did the trick (don't know how to call void in cljs)#2023-04-2820:22vincentcreative!#2023-04-2821:42Dustin Getzto reset the class, perhaps drop out of the declarative style and use a fn (touch-dirty-animation x) which explicitly unmounts and remounts the class (thus letting you leave it there?)#2023-04-2914:15Dustin Getza delay operator can be implemented with missionary, delay is a discrete time concept. i believe it is (new (m/eduction (drop 1) (e/fn [] x))) where x is an electric reactive val#2023-04-2914:18Dustin Getze/fn here is being used to add a layer of flow (like a promise of a promise), passing it to missionary is weird, what’s happening is we’re taking advantage of knowledge that electric compiles e/fn down to missionary flows, so it typechecks in the end#2023-04-2916:49xificurCThe delay operator above wouldn't work in electric since everything has to have an initial value (we're in continuous time). There could be an implementation that throws pending first or provides a default first value#2023-04-2918:59Dustin Getz(new (->> (e/fn [] x) (m/eduction (drop 1)) (m/reductions {} nil))#2023-04-2919:01denikfirst get a reactive version of the closure this fn returns:
(defn changed-fn []
(let [!prev (atom ::none)]
(fn [x]
(let [p @!prev]
(reset! !prev x)
(and
(not= ::none p)
(not= p x))))))
then compose it with something that always returns the changed-fn’s return value when a new one is available but then returns a default value after a delay#2023-04-2919:02Dustin Getzyou can use an atom to detect the edge, but i think in the end it is equivalent to the (drop 1) formulation#2023-04-2919:04deniknot understanding how (new (->> (e/fn [] x) (m/eduction (drop 1)) (m/reductions {} nil)) would allow to compare previous and current value of x and turn false after a delay#2023-04-2919:05denikthe goal is to have an expression that looks like
(when (changed-or-delay? x 500) "css-class")
#2023-04-2919:21Dustin Getzusage is (= x (delay x))#2023-04-2919:22Dustin Getzyou could use that to touch the class by side effect, presumably the css animation will handle the fade out #2023-04-2919:24deniknot sure I understand. what is a possible implementation for:
(when (changed-or-delay? x 500 false) "css-class")
#2023-04-2919:26denikseems like changed-or-delay? could be a flow and addition/removal of the class would be taken care of by electric#2023-04-2919:27denik(e/defn changed-or-delay [x delay on-delay-value] <impl>)#2023-04-2919:43Dustin Getzaren’t you going to do a css fade? and if so the 500ms constant is specified in css?#2023-04-2919:44denikit’s not a fade, it’s an animation that runs once#2023-04-2919:44deniki just need the delay to be the duration of the animation so that it is removed after running#2023-04-2919:45Dustin Getzwhy do you need to remove it?#2023-04-2919:46denikbecause otherwise it won’t run again#2023-04-2919:46denikcss :man-shrugging:#2023-04-2919:47Dustin Getzyou can remove it right before readding#2023-04-2919:47denik#2023-04-2919:48denikI have a pretty good idea of what is needed and think the most elegant way to achieve it is a flow of the shape described above#2023-04-2919:49xificurC(let [cl (.-classList dom/node)]
(keep-for 500
(do (.add cl "myclass")
(e/on-unmount #(.remove cl " myclass")))))#2023-04-2919:49denikcould you help me implement it? it’s a quite widespread use case and having a solution would come in handy#2023-04-2919:50Dustin Getzi believe i’ve given you the simplest solution #2023-04-2919:51denikI don’t understand how that solves it#2023-04-2919:51Dustin Getzi didn’t h understand the snippet i can try again another time if you guys don’t get it sooner #2023-04-2919:51denik#2023-04-2919:52denikhere’s the rest of the code. everything but the changed-or-delay? part works great already. feels idiomatic to just have it there instead of manipulating the classlist directly#2023-04-2919:55Dustin Getzthe api you’re asking for is more complicated, i don’t have an easy answer off the top of my head#2023-04-2920:13xificurCmaybe? There's some difference in the two, changed? will return true for a single propagation step; it will "blink" true then turn back to false immediately. (after 500 false true) will return false for 500ms then true forever.#2023-04-2920:23denikthe mechanics of that are not quite right#2023-04-2920:23denikbasically want a https://en.wikipedia.org/wiki/Useless_machine#2023-04-2920:23xificurC#2023-04-2920:24denikupdated-at value changes: TRUE
500ms pass: FALSE
repeat#2023-04-2920:28xificurCthe above might work. In continuous time the switch might be missed, not sure. That's why discrete events are still often handled imperatively#2023-04-2920:38denikit works once but not again#2023-04-2921:01denikwould be great if this worked
(when (keep-for 1000 (changed? updated-at) false)
"animate-[shadow-pulse_1s_ease-in-out] z-10")#2023-04-3018:44xificurCI think this pattern is unsafe and you should use an effect.
(defn animate [node css-class]
(let [cl (.classList node)]
(.remove cl css-class)
(set! (.-offsetWidth node)
(.-offsetWidth node))
(.add cl css-class)))
No need to remove after delay, this will re-trigger it.
Add an extra unused arg to pass updated-at and it will auto-rerun on change. No need for the change machinery, just leverage the reactive graph directly.#2023-04-3018:47denikand offsetWidth needs to be set to rerun the css?#2023-04-3018:48xificurCWhy are the previous patterns unsafe? Electric is a continuous time abstraction which can skip values and just use the latest value. A quick remove and add might go unobserved.#2023-04-3018:49xificurChttps://css-tricks.com/restart-css-animation/ talks about the offsetWidth hack and it worked for me locally (linux firefox)#2023-04-3019:06denikthanks @U09FL65DK. one issue with the impl above is that it runs on every first render instead of only on change. It could be wrapped it in a function with an atom switch to avoid the initial run. I’m still a somewhat surprised that the solution ends up feeling so hacky :thinking_face:#2023-04-2916:44mihaelkonjevicUsing electric clojure with a non reactive db#2023-04-2916:46mihaelkonjevicHi everyone. I have some questions about how to best use electric clojure with a non reactive database (PostgreSQL). I’ve seen this gist - https://gist.github.com/dustingetz/1960436eb4044f65ddfcfce3ee0641b7- by @U09K620SG which was very helpful, but I’m trying to come up with an approach that would be more appropriate for production usage.
This is what I’ve come up so far, but I’d like to get some feedback, because although it works, I’m not sure it works correctly, and if there are ways to improve it.
The idea is to use Redis Pub/sub mechanism to invalidate views whenever a change happens. So, each mutation would publish a message to Redis, and then components could subscribe to these messages and re-render if needed. I’ve managed to make it work in a pretty granular way, but this was my first time diving into missionary, so I’d like to get any feedback on how to improve the code. Here’s the code, it’s based on the starter app, and although it’s using datascript, I’m using it in a non-reactive way, because in real app it would be replaced with PostgreSQL
(ns app.todo-list
(:require contrib.str
#?(:clj [datascript.core :as d]) ; database on server
#?(:clj [taoensso.carmine :as car])
#?(:clj [missionary.core :as m])
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
#?(:clj (def redis-spec {:uri ""}))
#?(:clj (defonce redis-connection-pool (car/connection-pool {})))
#?(:clj (def wcar-opts {:pool redis-connection-pool :spec redis-spec}))
#?(:clj (defonce !conn (d/create-conn {}))) ; database on server
(e/def user-id)
(e/def publisher)
#?(:clj
(defn observe-redis [user-id]
(m/observe (fn mount-sub [!]
(let [key (str user-id ".*")
listener (car/with-new-pubsub-listener (:spec wcar-opts)
{key (fn [[_ _ topic payload]]
(let [action (second (str/split topic #"\."))]
(! [{:action action :payload payload}])))}
(car/psubscribe key))]
(! [])
(fn unmount []
(car/close-listener listener)))))))
#?(:clj
(defn publish-message [key todo]
(car/wcar wcar-opts (car/publish (str key) todo))))
#?(:clj
(defn get-entity [!conn id _sub]
(let [db @!conn]
(d/entity db id))))
#?(:clj
(defn todo-count [!conn user-id _sub]
(let [db @!conn]
(count
(d/q '[:find [?e ...] :in $ ?status ?user-id
:where [?e :task/status ?status]
[?e :user/id ?user-id]] db :active user-id)))))
#?(:clj
(defn todo-records [!conn user-id _sub]
(let [db @!conn
records (->> (d/q '[:find [(pull ?e [:db/id :task/description]) ...]
:in $ ?user-id
:where [?e :task/status]
[?e :user/id ?user-id]] db user-id)
(sort-by :task/description))]
records)))
(defn make-matching? [matchers]
(fn [item]
(reduce
(fn [_ matcher]
(if (matcher item)
(reduced true)
false))
false
matchers)))
(defn filter-matching-events [matchers]
(fn [batch]
(->> batch
(filter (make-matching? matchers))
seq)))
(e/defn GetMatchingEvents [matchers]
(let [!prev (atom nil)
prev (e/watch !prev)
matching (new
(m/relieve
mapcat
(m/eduction
(map (filter-matching-events matchers))
(m/subscribe publisher))))]
(when matching
(reset! !prev matching))
prev))
#?(:clj
(defn toggle-item [!conn user-id id status]
(let [key (str user-id ".todo-updated")]
(d/transact! !conn [{:db/id id
:task/status status}])
(publish-message key {:id id :status status}))))
(e/defn TodoItem [!conn id]
(e/server
(let [sub (GetMatchingEvents. [#(and (= "todo-updated" (:action %))
(= id (get-in % [:payload :id])))])
e (get-entity !conn id sub)
status (:task/status e)]
(println ">>>>" e)
(e/client
(dom/div
(ui/checkbox
(case status :active false, :done true)
(e/fn [v]
(e/server
(e/offload #(toggle-item !conn user-id id (if v :done :active)))
nil))
(dom/props {:id id}))
(dom/label (dom/props {:for id}) (dom/text (e/server (:task/description e)))))))))
(e/defn InputSubmit [F]
; Custom input control using lower dom interface for Enter handling
(dom/input (dom/props {:placeholder "Buy milk"})
(dom/on "keydown" (e/fn [e]
(when (= "Enter" (.-key e))
(when-some [v (contrib.str/empty->nil (-> e .-target .-value))]
(new F v)
(set! (.-value dom/node) "")))))))
(e/defn TodoCreate []
(e/client
(InputSubmit. (e/fn [v]
(e/server
(let [key (str user-id ".todo-created")]
(d/transact! !conn [{:task/description v
:task/status :active
:user/id user-id}])
(e/offload #(publish-message key v)))
nil)))))
(e/defn TodoListItems [user-id]
(e/server
(let [sub (GetMatchingEvents. [#(= "todo-created" (:action %))])]
(e/client
(dom/div {:class "todo-items"}
(e/server
(e/for-by :db/id [{:keys [db/id]} (todo-records !conn user-id sub)]
(TodoItem. !conn id))))))))
(e/defn TodoListCount [user-id]
(e/server
(let [sub (GetMatchingEvents. [#(= "todo-created" (:action %))
#(= "todo-updated" (:action %))])]
(e/client
(dom/p (dom/props {:class "counter"})
(dom/span
(dom/props {:class "count"})
(dom/text
(e/server
(todo-count !conn user-id sub))))
(dom/text " items left"))))))
(e/defn TodoList [search]
(e/server
(let [user-id (subs (or search "?0") 1)
observer (m/relieve mapcat (observe-redis user-id))]
(binding [user-id user-id
publisher (m/publisher observer)]
(e/client
(dom/h1 (dom/text "minimal todo list!"))
(dom/p (dom/text "it's multiplayer, try two tabs"))
(dom/div (dom/props {:class "todo-list"})
(TodoCreate.)
(TodoListItems. user-id)
(TodoListCount. user-id)))))))
• Is this a proper way to bridge Carmine and Electric Clojure (I’m using the pattern I saw in the implementation of router)?
• Is this a proper way to do subscriptions? I want to have one connection to redis, and then every component should be able to filter any relevant messages
• Is there a better way to implement GetMatchingEvents? It’s using an atom to store last relevant message batch, so it doesn’t cause re-renders when an irrelevant message comes in
• I’m using m/relieve with mapcat, from what I understand, this will batch messages if backpressure is needed. Is this correct?#2023-04-2918:37vincentsounds like you really need postgres. it sounds like a mess to me honestly, but it is possible i reckon. yeah you need some sort of propagation of changes back to the client, right? (@ hyperfidle team)#2023-04-2918:57Dustin GetzHave you looked into the pg_notify API? Another strategy is to refresh queries on navigate by integrating the router, and make only certain views realtime using either pg_notify or a pub/sub like approach as you outlined#2023-04-2919:01Dustin Getzi will try to do a deeper code review later in the week#2023-04-2919:33mihaelkonjevic@U04PBP0MYS3 PostgreSQL feels like a great, safe default choice. Datomic might be an option, especially now it’s free, but it definitely comes with some extra deployment complexity.
@U09K620SG pg_notify might be an option, but it could also introduce messy layer in which I have to determine which views to refresh based on the data changes. With redis pub/sub, I can use topics to communicate “intent” or source of the change which seems like it might simplify the invalidation layer.#2023-04-3000:32Dustin Getzi don’t see how a command pattern solves the messy query refresh problem, in the worst case someone can make a one off edit via pgadmin gui and your queries need to refresh, in the best case where you just promise to only write via commands you can probably inspect the transaction to decide which views to query#2023-04-3009:49mihaelkonjevic@U09K620SG all of that makes sense. I’ll definitely look more into pg_notify #2023-05-0410:45braai engineer@U050986L9 look at Materialize https://materialize.com/#2023-04-2923:41markaddlemanI spent a couple of hours putting together a POC of electric hiccup. https://gist.github.com/markaddleman/a1f5a5179d79df6e6baa19c0184d2d0b#2023-04-2923:42markaddlemanAs I got through this exercise, I began to realize there probably isn’t much point but I’m curious to hear other people’s thoughts.#2023-04-2923:42markaddlemanThe one good thing is that I have a much better understanding of the electric DSL.#2023-04-3011:36Dustin Getzfyi - https://github.com/hyperfiddle/electric/issues/32#2023-04-3011:37Dustin Getzhappy to discuss more here to flesh out that issue with more design Q&A and collect previous attempts, there are some threads here as well #2023-04-3011:37Dustin Getzi’m curious what exactly motivated you to attempt this(i can think of a few possible reasons)#2023-04-3011:57markaddlemanHi Dustin. I had a few motivations and they evolved along the way:
• I thought it would be a good way to understand electric’s DSL
• Get a little bit into electric’s internals
• Appreciate the difference between electric’s declarative approach and hiccup / clojure’s functional approach#2023-04-3012:04markaddlemanUltimately, my motivating factor is cultural: I have a team of Clojure newbies who are falling in love with Clojure’s data oriented style and that they can solve almost any problem with the proper combination of map , reduce, and filter. I also need to build lots of UI quickly.
Electric is a new language with a different set of biases than Clojure. I began thinking about the pedology of my problem and realized I would spend a bunch of my time saying things like, “That doesn’t work here. Remember, you’re in Electric-land not in Clojure-land.” In an effort to reduce that problem, I wondered if hiccup would allow of a smoother on-ramp for developers moving from Python->Clojure->lots of UI.#2023-04-3012:05markaddlemanSaid another way, I was trying to explore the question, does hiccup reduce the seams between Clojure and Electric?#2023-04-3012:09markaddlemanAs a result of this exercise, I think I agree with the @U050CJFRU’s comment “Hiccup might be great for serialization (which doesn’t make much sense in electric’s case anyway)” (https://github.com/hyperfiddle/electric/issues/32#issuecomment-1478393433)#2023-04-3019:08denik@U2845S9KL looks like you arrived at a similar conclusion. Yes, as good as data is, code that’s data is better 🙂#2023-04-3020:19markaddlemanI was struggling to remember how to use zippers for a bottom up execution of the hiccup tree when I realized this whole exercise was probably pointless 😂#2023-05-0621:30marciolInteresting thread, I was thinking about electric these days and wondering if it'd be good to have hiccup as a way to capture more the attention of developers who are already familiarized with hiccup through reagent and other libraries using this syntax.#2023-05-0622:12Dustin Getzwe will add hiccup, or something like it, eventually. in the meantime we haven’t actually figured out a great way to do it that doesn’t damage electric’s value prop in other ways#2023-04-3009:58Lidor CohenHello again,
I'm looking to convert a self-ref map that represents a state graph to a DAG structure that electric will be able to consume easily, I know you've worked with something like this under the hood, except for the self-ref part but that's on me, I'm just looking for some target representation of a DAG, can you recommend anything?#2023-04-3010:23Dustin Getztell me more about the problem you’re solving so i can understand the use case?#2023-04-3010:35Lidor CohenWe have nested structures that looks like this:
{:name ""
:greeting (str "Hello, " (this :name))}
*this is a simple example of course.
It can be compiled to something like this:
(let [!name (atom "")
name (e/watch !name)
greeting (str "Hello, " name)]
...)
This will require us to do ourselves the traversals and manage the dependencies.
So I was wondering if you know and data-structure in clojure that can handle this kind of self-ref or at least a good target to compile our data to it and from there to electric#2023-04-3011:32Dustin Getzi need to think about it. we have pretty much this exact structure inside HFQL, we analyze it to datascript and then query datascript to emit electric code#2023-04-3011:33Dustin Getzperhaps HFQL itself solves your business problem, if you zoom out a level or two what are we trying to accomplish with these maps?#2023-04-3011:44Lidor Cohenit's a bit high level, and out of context it might lose its meaning, but I guess it comes down to expressive and reusable state management#2023-04-3011:45Lidor Cohenthis aspect (self-ref) resembles mobx a bit#2023-04-3011:46Lidor Cohendatascript was one of my first hunches too, I started experimenting with it.#2023-04-3011:54Dustin Getzlike are you rendering forms, or what#2023-04-3012:00Lidor CohenMy POC is a simple form, yes:
(e/defn Input-example []
(e/client
(let [!name (atom "")
name (e/watch !name)
greeting (str "Hello, " name)]
(dom/input (dom/on "input" (e/fn [e] (swap! !name #(-> e .-target .-value))))
(dom/props {:value name}))
(dom/p (dom/text greeting)))))
This is my first target from the state in the example above#2023-04-3012:06Dustin GetzHFQL may be the exact thing you need, we haven’t published it yet but there are some screenshots here https://hyperfiddle.notion.site/Hyperfiddle-progress-update-Feb-2023-8cc45f9da47c4719bb16851d129e3a3d#2023-04-3012:20Lidor CohenI think HFQL is too high level, I'm trying to handle only state (abstract data) for now.
How did you break arbitrary maps into datascript? if I'm not specifying a ref in the schema it just takes the maps as a whole value. is there a way to tell datascript to break arbitrary maps to EAVs?#2023-04-3015:05xificurCRoughly speaking HFQL extends EQL. It doesn't try to manage state but express a view#2023-04-3018:40Dustin GetzIn the example you gave
{:name ""
:greeting (str "Hello, " (this :name))}
IMO the "map" here seems to be the desired result {:name "Lidor" :greeting "Hello, Lidor"} , which is a projection/query over an underlying graph#2023-04-3018:40Dustin GetzI hear that you think HFQL is too high level, but if I may show you one more thing#2023-04-3018:42Dustin GetzThis is how I'd produce basically the above map using HFQL and a datascript graph:
(defn greeting [name_]
(str "Hello, " name_))
(e/defn Form [id]
(hf/hfql
[:person/name
(greeting person/name)]
id))
(comment
(Form. 1) :=
{:person/name "Alice"
'(greeting person/name) "Hello, Alice"})#2023-04-3018:42Dustin Getzyou can process it from there, we have an :alias directive as well if you need the key to be :greeting and not '(greeting id)#2023-04-3018:44Lidor CohenThat looks very much like the target I had in mind for that code, I still need to workout the details and how to work with datascript#2023-04-3018:46Lidor CohenBut basically:
Traverse graph
Primitives turn to values in ds
Computations turn to functions with pointers to dependants#2023-05-0116:28Dustin Getzedited the gist to be simpler and slightly more powerful with a sideways lookup#2023-05-0209:04Lidor Cohen(defn greeting [id]
(str "Hello, " (d/entity db id)))
(e/defn Form [id]
(hf/hfql
[:person/name
(greeting id)]
id))
(comment
(Form. 1) :=
{:person/name "Alice"
'(greeting id) "Hello, Alice"})
This one?#2023-05-0213:05Dustin Getzyeah i edited that, it uses ‘(greeting person/name) now#2023-04-3014:19markaddlemanIt seems like the value of electric-lang vars is nil is clojure-land.
(e/defn E [])
(def m {"k" E})
(println m)
Is this expected?#2023-04-3014:22markaddlemanThis came up as I was trying to invoke an Electric function from a Plotly event callback. Very roughly, the callback code looks like
(fn [e] (new ElectricFn e))
Since this code lives in clojure-land, ElectricFn is nil and, thus, the code fails#2023-04-3014:23markaddlemanWell, I’m a bit of a dummy. I just noticed the clojurescript compiler complaining {k #object[clojure.lang.Var$Unbound 0x63bc661f Unbound: #'app.plotly-interop/E]}#2023-04-3015:21Dustin Getzthis is internals and could change. today, e/def quotes the body and saves it as metadata for a later compiler phase. electric things are only visible from inside/under the electric entrypoint#2023-04-3015:22Dustin Getzin (fn [] (new ElectricFn)), new is clojure new not electric new because it’s in a clojure context#2023-04-3015:23markaddlemanFor interop reasons, I think it would be helpful to have access to the Electric new#2023-04-3015:27Dustin Getzpost the whole plotly integration pls, we will find a clean way to bridge them#2023-04-3015:29markaddleman#2023-04-3015:30markaddlemanMy strategy to send plotly events into electric-land is by dispatching events to the surrounding div where I’ve added dom/on reactions#2023-04-3015:31Dustin Getzif you look at our codemirror integration in the repo you might be able to figure out the pattern, the idea is to bridge the event to a discrete missionary flow immediately and then adapt to electric from there#2023-04-3015:31markaddlemancool. Can you point me to that? I don’t remember seeing a codemirror integration in the tutorials#2023-04-3015:32Dustin Getzon mobile you’ll need to grep the main repo#2023-04-3015:32markaddlemanno worries. thanks!#2023-04-3017:12Dustin Getzthis constructs a codemirror editor, registers a value change listener, and redirects change events into a missionary discrete flow >cm-v with m/observe. >cm-v which is then parsed (e.g. string to edn) and given an initial value before bridging to electric with new. https://github.com/hyperfiddle/electric/blob/faab6602ee84361d81f24e31609f01b5748a3d6f/src/contrib/electric_codemirror.cljc#L73-L83#2023-04-3017:14Dustin Getzm/observe is the key primitive that adapts an event listener callback interface to a discrete flow, note m/observe also forces you to define the cleanup code in the same place to unregister the callback and destroy the codemirror (which missionary will call during teardown)#2023-05-0113:44markaddlemanI stared at this code for a while and I have low confidence I understand about 30% and high confidence that I don’t understand 70% of it 🙂 My current approach is working: reflect events onto a div on which I can place an electric on and use e/fns from there.#2023-05-0113:45markaddlemanIs there an issue for improving the ergonomics of javascript->electric interop?#2023-05-0113:55Dustin GetzI think the real issue here is that the codemirror API is more complex than it seems which is causing contorted code to functionalize it#2023-05-0116:40Dustin GetzBased on my understanding of the plotly API (which is redirecting events through the div it attaches to) i think this is sufficient:
(e/defn Plotly [plotly-config]
(let [plotly-id (str (random-uuid))]
(dom/div (dom/props {:id plotly-id})
(plotly/newPlot plotly-id (clj->js plotly-config))
(dom/on! "plotly_click" (fn [data]
(let [curve-number ^int (. (aget (. data -points) 0) -curveNumber)
point-number ^int (. (aget (. data -points) 0) -pointNumber)]
[curve-number point-number]))))))
(comment
(let [plotly-click-data (Plotly. plotly-config)]
; electric body
...))#2023-05-0116:41Dustin GetzWe could tighten it up by using m/observe to separate the plotly lifecycle from the div's lifecycle (allowing you to remove the plotly but keep the div) but it doesn't seem to matter here#2023-05-0116:42Dustin GetzAs to making this easier to figure out, its unclear what's possible, i need to see a lot more of these and maybe there is some sort of common pattern, or also possibly each JS integration is a unique snowflake. This guy was a bit of an oddball as well#2023-05-0119:20markaddlemanYour code snippet doesn’t work because (I believe) of some plotly wierdness. From what I can tell, plotly events are not real javascript events. Plotly attaches an on function to its div and the client must call that to attach listeners. Making it more complicated, plotly/newPlot returns a JS promise which, on resolution, returns the div to which its on function is attached.#2023-05-0119:20markaddlemanthus, my entire Plotly-Chart electric function looks like:
(e/defn Plotly-Chart [plotly-config event-Reactions]
(e/client
(let [id (str (random-uuid))]
(dom/div
(doseq [[event Reaction!] event-Reactions]
(dom/on event Reaction!))
(dom/props {:id id}))
(.then (plotly/newPlot id (clj->js plotly-config))
(fn [div]
(doseq [[event _] event-Reactions]
(.on div event
(fn [data]
(let [event (new js/Event event)
curve-number ^int (. (aget (. data -points) 0) -curveNumber)
point-number ^int (. (aget (. data -points) 0) -pointNumber)]
(set! (.-data event) [plotly-config curve-number point-number])
(.dispatchEvent div event))))))))))
#2023-05-0119:28Dustin Getzok i’ll look closer at that .on synthetic thing. we already have the div, do we need to wait again?#2023-05-0119:37markaddleman> do we need to wait again?
Not sure what you mean by this.
Right now, I’m reflecting the plotly data (available from its weird callback) as a real javascript event to the electric div. My ideal would be to take the plotly data object and pass it directly to a electric fn. Something like the following code:
(e/defn Plotly-Chart [plotly-config event-Reactions]
(e/client
(let [id (str (random-uuid))]
(dom/div
(dom/props {:id id}))
(.then (plotly/newPlot id (clj->js plotly-config))
(fn [div]
(doseq [[event Reaction] event-Reactions]
(.on div event
(fn [data]
(new Reaction data)))))))))
#2023-05-0119:38markaddlemanObviously this doesn’t work because (as you pointed out earlier), the (new Reaction data) is not an electric new.#2023-05-0119:39markaddlemanIt occurs to me that a real implementation would also have to convert data to edn before creating the Reaction object because it might have to be sent over the wire.#2023-05-0117:40andersmurphyWhich of the the models presented in this blog post if any best describes electrics rendering/state model?
https://tonsky.me/blog/humble-state/
electric doesn’t have a vdom right? It’s reactive incremental computations?#2023-05-0118:05Dustin Getzat a quick glance, we match the phrase “surgical point updates” and “incremental” - however i cannot endorse the accuracy of the post or any claims it makes or implies about Electric#2023-05-0120:18markaddlemanAre the semantics of Pending supposed to match a real exception? The following code produces non-intuitive results:
(try
(new (e/fn [n]
(let [n (e/server (Thread/sleep n) n)]
(e/client (dom/text (str "loaded after " n "ms")))))
1000)
(println "loaded")
(catch Pending _
(println "waiting...")))
#2023-05-0120:19markaddlemanOn the console, I see “loaded” followed immediately by “waiting…”#2023-05-0120:20markaddlemanMy desired behavior is for the catch block to be executed and, when e/fn completes, I’d see “loaded”#2023-05-0120:20markaddlemanIs there an idiomatic means to force the desired behavior?#2023-05-0120:39noonianI'm not sure about idiomatic, but I think one way to do it is to make the expression that prints "loaded" use the return value of calling the photon fn. You could do this by writing a macro that just discards the first arg value and returns the second then use that to call println.
I believe the behavior you are seeing is because the "println" expression and the "new" expression are effectively executing in parallel which is why you see "loaded" print immediately#2023-05-0120:40markaddlemanYeah, I was just experimenting with a similar approach. In the real case, the e/fn can return a value or nil so I end up writing some silliness. Fortunately, the electric compiler does not try to short-circuit expressions :)
(try
(let [v (new (e/fn [n]
(let [n (e/server (Thread/sleep n) n)]
(e/client (dom/text (str "loaded after " n "ms")))))
1000)]
(when (or v (not v))
(println "loaded")))
(catch Pending _
(println "waiting...")))
#2023-05-0120:48noonianYeah similar idea. I'm curious, does it do what you want if you just have (do v (println "loaded"))? I suspect it tries to parallelize do blocks but I'm not sure. v won't have a value until the function executes but I'm not at my computer atm to check#2023-05-0120:52xificurCdo indeed runs the branches concurrently
To wait for a Pending to resolve for now we sequence with case.
(case a b)
b will run when a won't be exceptional#2023-05-0120:54xificurCThread.sleep blocks the main thread and therefore the whole graph. You can use a missionary sleep to wait and not block the world
(case (new (e/task->cp (m/sleep n)))
b)
#2023-05-0121:03markaddlemanThanks. Of course, this was all for demo purposes but it’s good to know that m/sleep exists#2023-05-0121:12Dustin Getzthink of it this way -
(do (dom/div) (dom/div)) will mount both divs. think of them as components not statements. if they contain reactive dependencies then they will be maintained concurrently. Note that (do (println 1) (println 2)) will mount the printlns in order (and then never touch them again as the exprs are constant), so this is backwards compatible with clojure#2023-05-0206:45xificurCthe ordering is not yet guaranteed btw. It works and probably won't change. Cancellation order we're discussing too#2023-05-0206:47xificurCre demo purpuses - sure. I just wanted to point it out since it can trip up demos where you expect some server reactivity#2023-05-0206:28gravHow come there's a special e/for form? I read through the examples and I couldn't really find any explanation, apart from it being "reactive". But then I'm wondering why the sequence itself couldn't be reactive instead, so that the usual for, map, filter etc could be used?#2023-05-0206:40gravI'm guessing, looking at https://electric-examples-app.fly.dev/user.demo-system-properties!%53ystem%50roperties, that e/for[-by] is only necessary at the edges, so it's probably not a huge inconvenience.#2023-05-0206:43xificurCa reactive sequence implies a new data structure, right?#2023-05-0206:47gravI guess so, but couldn't it still implement clojure.lang.ISeq or whatever's necessary to support the core fns?#2023-05-0206:49xificurCe/for has the same use case as e.g. react keys - efficient (diffed) sequence handling. E.g. if a query returns 10 elements and you mount 10 <li>s and the next query returns 9 of the same elements you want to reuse the 9 <li>s you already mounted#2023-05-0206:52xificurCre reactive data structures - probably, but then you'd need to convert to this data structure at the edges (reactive-seq (query-system-properties needle))#2023-05-0206:55xificurCalso note that fine grained reactivity wants you to break collections down to the smallest elements ASAP. Ideally a database would stream changes of your query reactively and you could skip the diffing entirely#2023-05-0212:53Dustin Getzthe simple answer to why there is a special e/for form is that today the e/for semantics differ from clojure.core/for in several ways - in addition to diffing by react key (arity change), e/for is also eager whereas cc/for is lazy#2023-05-0212:58Dustin Getzseparately, as to cc/map, cc/filter etc- we retain clojure compatibility, so we are careful to not change the semantics of clojure core operators. map and filter rebuild the list in clojure so they do the same in Electric. They also take clojure lambdas as an argument not e/fn. There are other possible designs, the one we chose is intended to maximize backwards compatibility with clojure so that pre existing macros (like —> , when-not, all still work - including complex macros like missionary)#2023-05-0213:03Dustin Getzfinally, at some point you start to ask if the data structures themselves should become incremental, which is an open research problem. a lot of reactive language research gets stuck yak shaving here. we chose to use ordinary clojure data structures directly, which means assoc, get-in, etc are all ordinary (not incremental). this is another reason to take the tradeoff we made#2023-05-0213:14gravCool, thanks for explaining your considerations @U09K620SG!#2023-05-0213:13markaddleman(e/defn Main []
(js/setTimeout #(println "here") 1000))
When new’d up, I get
<unknown interop> {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/args (#object[G__78921] 1000), :hyperfiddle.electric.debug/origin #uuid "e2836829-54b0-41fc-8fbe-bb22dd6047fc"}
in reactive (defn Main [] ...) in app/plotly_interop.cljs line 117
in (try ...)
#2023-05-0213:14markaddlemanMy goal is to refresh charts on a dashboard every n seconds. I looked at e/system-time but that seems to run far more frequently than I need. I only need to refresh every hour#2023-05-0213:33Geoffrey GaillardYou can truncate e/system-time-ms to hours. Electric will efficiently work-skip consecutive values.
(e/def system-time-hours "hours since 1970 Jan 1"
(js/Math.floor (/ system-time-ms 3600000)))
#2023-05-0213:35Geoffrey GaillardOtherwise, (.setTimeout js/window #(…) …) should work#2023-05-0213:42Dustin GetzI want to reinforce what Geoffrey said wrt "I only need to refresh every hour" - this imperative mindset is an anti-pattern. See https://electric.hyperfiddle.net/user.tutorial-backpressure!Backpressure#2023-05-0214:21markaddlemanI’m sure there’s a ton that I don’t understand but, naively, the underlying electric clock is running at 120hz so there is some browser-side CPU work happening pretty frequently even if very little because of the Electric optimizations.#2023-05-0214:21markaddlemanI’m pretty new to UI programming so maybe this is just how things are done in that browser environment#2023-05-0215:59Dustin GetzFYI mark and I did a zoom call#2023-05-0217:37JHi! I’m asking if Datomic can act like xtdb v1 (in term of reactivity)? I mean you get a connection and watch the db snapshot. I don’t know if it’s possible with datomic#2023-05-0219:43Dustin GetzYes it's possible at least in Datomic Onprem, you need to integrate the tx-report-queue API#2023-05-0219:44Dustin Getzthe impl should look similar to the XTDB version#2023-05-0219:44Dustin Getzhttps://docs.datomic.com/pro/clojure/index.html#datomic.api/tx-report-queue#2023-05-0303:38JThanks 🙏 #2023-05-1514:00JFirst try => https://gist.github.com/jeans11/845fdeedbf1257a8a3355d03f6bd4b31
Feel free to update. I didn’t have find a way to use the observe function like xtdb. Missionary is realy cool!#2023-05-1515:40Dustin GetzPost in #CL85MBPEF where Leo will see it and you might get a code review from him 🙂#2023-05-0300:26VincentHey, I figured out my websocket issue. Turns out you can proxypass everything to whatever port the server is running and it should just work 😄
So that's good!
So now I just need to figure out how to retroactively add session support to the electric-starter-app#2023-05-0300:40Dustin Getzwhat specifically do you mean by “session state”, is there a framework you are referring to?#2023-05-0300:58VincentJetty / Ring I guess#2023-05-0300:58Vincent/auth endpoint from the examples#2023-05-0300:58Vincentnot exactly sure which examplse that is in...#2023-05-0301:02Dustin Getzah that used a cookie#2023-05-0301:09VincentAh cool#2023-05-0300:27Vincentthen I think, a way to handle notifications & emailing stuff [via SparkPost]
and a "nice to have" would be how much time users spend on pages and a "heatmap" for where the mouse is / what links they click. I wonder if it makes sense to work on an open-source version of that for the community#2023-05-0300:59Dustin Getzi think there are js saas analytics tools that do the heat map stuff#2023-05-0301:00Dustin Getzwhen i think of notifications i think mostly of mobile apps and slack notifications, are you making a mobile app?#2023-05-0301:03Vincentcommunity musician place, so sorta#2023-05-0301:05Vincentyeah maybe can do heat maps with after party tool, just figured with a web socket already there, probably a lot that can be done#2023-05-0314:50markaddlemanShould metadata survive a transport between client and server?#2023-05-0314:53Dustin GetzI believe so, have you encountered an issue? We transport with transit#2023-05-0314:53markaddlemanI don’t think it does. I’ll have a repro case in a minute (or I will have my bug 🙂 )#2023-05-0314:58markaddleman(e/defn Main []
(e/client
(let [m (e/server (with-meta {:key "value"}
{:metadata true}))]
(println m (meta m)))))
#2023-05-0314:59markaddlemanthe console shows {:key value} nil#2023-05-0314:59Dustin Getzi logged a ticket to look into it#2023-05-0315:00Dustin Getzwhat are you using this for btw?#2023-05-0315:02markaddlemanMy use case:#2023-05-0315:05markaddlemanI have a server-side function that makes a rest call. If the rest call fails, I catch the exception on the server side and create an error map and return that error map to the client as a return value (not an exception). The error map has metadata on it that allows the client to know if the value it receives is an error or not.#2023-05-0315:05markaddlemanAs I write this, I realize I should try throwing an ex-info object from the server to the client.#2023-05-0315:07Dustin GetzYeah, we will look into fixing it, a workaround would be to unpack the metadata in some way (into an ex-info or just a tuple)#2023-05-0315:08markaddlemanbtw, after a quick experiment, it doesn’t look like ex-info objects can be transmitted from server to client. I get error
java is not defined
in <unknown interop> {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/args (nil), :hyperfiddle.electric.debug/origin #uuid "2c01b195-c85a-4166-9d01-b26d44503c23"}
#2023-05-0315:08Dustin Getzok, please paste the snippet you used#2023-05-0315:13markaddlemanMy simple repro case fails differently:
(e/defn Main []
(e/client
(try
(e/server
(throw (ex-info "Does this survive?" {:some "data"})))
(catch js/Error ex
(println (ex-message ex))
(println (ex-data ex))))))
The first println shows the “Does this survive?” as expected. The second println is a bunch of electric internal data:
{:hyperfiddle.electric.debug/trace [{: :, :hyperfiddle.electric.debug/meta {:file app/plotly_interop.cljc}, :hyperfiddle.electric.debug/type :apply, :hyperfiddle.electric.debug/name hyperfiddle.electric.impl.runtime/fail, :hyperfiddle.electric.debug/args [<exception>], :hyperfiddle.electric.debug/origin #uuid "15aa5f8b-d49b-4b6a-b69d-9c5a3d9139e9", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/meta {:file app/plotly_interop.cljc, :line 146, :column 7, :end-line 147, :end-column 63, :hyperfiddle.electric.debug/type :transfer, :hyperfiddle.electric.debug/name :hyperfiddle.electric/server}, :hyperfiddle.electric.debug/origin #uuid "15aa5f8b-d49b-4b6a-b69d-9c5a3d9139e9", :hyperfiddle.electric.debug/args [], :hyperfiddle.electric.debug/serializable true}], :hyperfiddle.electric/type :hyperfiddle.electric.debug/trace}
#2023-05-0315:14markaddlemanI’ll try to repro the java is not defined case later today#2023-05-0315:21Dustin Getzthis is probably just a bug, we may not have enough test cases for catching exceptions remotely#2023-05-0315:22markaddlemanHere’s the java is not defined repro case:
(e/defn Error [ex]
(dom/pre (prn-str ex)))
(e/defn Main []
(e/client
(try
(new (e/fn [] (throw (ex-info "Some error" {:key "value"}))))
(catch js/Error ex (new Error (ex-data ex)))))
#2023-05-0315:23Dustin Getznew Error#2023-05-0315:23Dustin Getzis sus#2023-05-0315:23Dustin Getzoh nvm#2023-05-0315:23markaddlemanYeah, I was just looking at that#2023-05-0315:23markaddlemanThis code does not error:
e/defn Main []
(e/client
(try
(new (e/fn [] (throw (ex-info "Some error" {:key "value"}))))
(catch js/Error ex (new (e/fn [] (ex-data ex))))))
#2023-05-0315:23Dustin GetzDoes Error shadow anything from js? js/Error#2023-05-0315:24markaddlemanYes, I think so#2023-05-0315:24Dustin GetzYeah I dunno if this is a bug or what, I will capture the ticket. I don't see signs of the original issue here#2023-05-0315:25markaddlemanOriginal issue being metadata? That’s right. These are two separate issues#2023-05-0315:25Dustin Getz"it doesn’t look like ex-info objects can be transmitted from server to client. I get error"#2023-05-0315:27markaddlemanThat’s correct. The problem turns out to be two separate things:
1. https://clojurians.slack.com/archives/C7Q9GSHFV/p1683126839339759?thread_ts=1683125429.355749&cid=C7Q9GSHFV
2. https://clojurians.slack.com/archives/C7Q9GSHFV/p1683127360862209?thread_ts=1683125429.355749&cid=C7Q9GSHFV#2023-05-0318:55markaddlemanI’m trying more javascript interop. This time with a library that depends on jquery. Specifically, I’m trying to translate this jsfiddle to electric http://jsfiddle.net/astrapi69/md1xztks/#2023-05-0318:57markaddlemanI’m not sure if my problem is related to electric or not. There’s a good changce that I’m just doing the interop incorrectly#2023-05-0318:59markaddlemanThis is my attempt so far. It’s failing with Uncaught TypeError: $(...).gridster is not a function#2023-05-0319:02xificurCtry just
(-> (js/$ ".gridster ul")
(.gridster (clj->js {:widget_margins [10 10]
:widget_base_dimensions [140 140]})))#2023-05-0319:05markaddlemanNow getting this
inst__30220__auto__.gridster is not a function
in ( .gridster (js/$ ".gridster ul") #js {:widget_base_dimensions #, :widget_margins #} )
in case default branch
in reactive (defn Main [] ...) in app/plotly_interop.cljc line 135
in (try ...)
#2023-05-0319:06xificurCif you comment the gridster interop out completely and open the web console can you call it from there with the js sample?#2023-05-0319:08markaddlemanagain with Uncaught TypeError: $(...).gridster is not a function#2023-05-0319:08markaddlemanIt’s as if something isn’t initializing correctly.#2023-05-0319:09markaddlemanI don’t understand jquery well enough to know how it’s supposed to work#2023-05-0319:10xificurCdid you do what the readme said
> Gridster is currently written as a jQuery plugin, so you need to include it in the head of the document. Download the latest release at http://jquery.com/.#2023-05-0319:10markaddlemanah, that’s probably the problem. Thanks for the pointer!#2023-05-0319:11xificurCmaybe it should go in package.json#2023-05-0319:11xificurCnot quite sure either, sorry#2023-05-0319:12markaddlemanYou gave me something to investigate - which is more than I had before!#2023-05-0322:37markaddlemanfyi, that was the issue. Thanks again for the pointer!#2023-05-0319:03gravIn the https://github.com/hyperfiddle/electric-starter-app, how come ws messages are passed when I edit the text in the input field (for each key press)?#2023-05-0319:04gravThere's nothing that indicates to me that the F function is called, except when Enter is pressed:
(dom/on "keydown" (e/fn [e]
(when (= "Enter" (.-key e))
(when-some [v (contrib.str/empty->nil (-> e .-target .-value))]
(new F v)
(set! (.-value dom/node) "")))))
(from https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L30)#2023-05-0319:18xificurCdom/on calls new on the supplied e/fn for each keydown event. new creates a new node in the graph. The graph is kept in sync between the client and the server#2023-05-0319:45gravAh, okay. I think I'm still not fully understanding why the server needs to know about this part of the graph on runtime#2023-05-0320:00xificurCany node can request data from the remote peer. At that point you need to know who is requesting it. Having the same graph means you can reference it directly.
As a theoretical optimization, if you can prove statically there will be no remote requests, one might be able to skip this step.#2023-05-0320:36Dustin GetzI endorse what Peter said, and will add that we haven't invested much in optimizations yet because we haven't seen any real world performance impact in our testing outside of 1-2 very specific cases that we're working to address. It's really only request/response waterfalls that damage performance as far as we are aware#2023-05-0320:37Dustin GetzObviously as the compiler matures there is a path to making this hyper-optimized#2023-05-0322:32VincentAre there alternate strategies to managing logins / logged-in state?#2023-05-0417:40Dustin GetzWe don't provide out of the box integrations for anything at this time, but happy to track requests#2023-05-0504:17VincentCool. Turns out probably totally unnecessary ^^ I think /auth is plenty, and I see it actually exists in all the examples I need x) I am just slow to the grip sometimes! x)#2023-05-0417:36markaddlemanIs there a story for vanilla javascript to construct and mount multiple electric components within a single dom?#2023-05-0417:36markaddlemanI have an existing javascript app and I’m thinking about how to incrementally electrify it.#2023-05-0417:37markaddlemanFrom what I can tell, I can boot multiple electric fns but only one can be hooked up to the reactor at a time.#2023-05-0417:37Dustin Getzyou could bind dom/node multiple times from the same boot#2023-05-0417:38Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/0fa1b265c1846c4a07db52dc1fdffa29ab729e65/src/user.cljs#L9C1-L10#2023-05-0417:38markaddlemanThanks, that makes sense#2023-05-0417:39Dustin GetzI don't immediately see any technical reason why you can't have multiple electric instances simultaneously but I think we would prefer you not do that#2023-05-0417:39Dustin Getzah, they will collide on trying to connect to the server#2023-05-0417:39markaddleman🙂 I feel like I’m about to do a bunch of things that someone is going to prefer I not do#2023-05-0417:39Dustin Getzhappy to walk with you on this journey#2023-05-0417:40markaddlemanLet me ping you back when I have some skeleton code that I can easily play with#2023-05-0500:48Alan CarrollVery new to all of this, just playing around and trying to see what I can do. For fun I'm trying to connect https://github.com/cjohansen/portfolio/tree/main with electric. I have a basic example working with just a header, working from the starter app logic:
;; app ns
(e/defn App []
(e/client
(portfolio.ui/start!)))
;; user ns
(def electric-main
(e/boot
(app/App.)))
;; portfolio scene ns
(e/defn Header []
(e/client
(dom/h1 (dom/text "Hello world"))))
(defscene header
(let [d (gdom/createElement "div")]
(e/run
(binding [dom/node d]
(Header.)))
d))
This results in a nice little header being shown. Huzzah! I next tried to see if I could recreate the two clocks demo, but I have failed miserably.
(e/defn Clock []
(e/client
(let [c (e/client e/system-time-ms)]
(dom/div (dom/text "client: " c)))))
(defdom clock
(let [d (gdom/createElement "div")]
(e/run
(binding [dom/node d]
(Clock.)))
d))
This displays ie client: 1683246954260 in the scene, but it never updates. Adding in (e/server e/system-time-ms) also simply prevents the scene from rendering at all, just a blank space. I've been naively stumbling around trying to make things work as I expect, but it's safe to say I have no idea what I'm doing. Any help understanding would be appreciated!#2023-05-0501:12Dustin Getzi think you want e/boot not e/run, and also call e/boot at the top level not inside some other reactive context#2023-05-0504:01Alan CarrollBy top level do you mean getting rid of the the (e/boot (app/App.)) layer and only issuing calls where I've used e/run? If so the portfolio UI loads and continues to run fine, so that's easy and fixed. However if I replace e/run with e/boot in the defscene forms then I run into some issues even with the simple Header case.
(defscene header
(let [d (gdom/createElement "div")]
(e/boot
(binding [dom/node d]
(Header.)))
d))
This fails to do anything. Looking through old chat it seems like e/boot is maybe supposed to be def'd and called, so I tried:
(defscene header
(let [d (gdom/createElement "div")
boot (e/boot
(binding [dom/node d]
(Header.)))]
(boot)
d))
Hot reload doesn't produce anything, but if I refresh the page then the header renders. However if I navigate to another scene and back, it doesn't re-render, and if I refresh the page on a different scene and navigate to it then it never renders. Fwiw, the Clock case is the same, but if I refresh the page on it so that it renders it also sits statically and doesn't update the time.#2023-05-0511:39Dustin Getze/boot should be called exactly once, e/run is a test entrypoint and should never be called #2023-05-0504:16VincentHey friends, I'm slamming a few examples together, and I came across this error
Unable to resolve symbol: e/*http-request*
{:file "app/todo_list.cljc", :in [e/*http-request*]}
ExceptionInfo: Unable to resolve symbol: e/*http-request*
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:643)
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:628)
hyperfiddle.electric.impl.compiler/eval30669/analyze--30675/fn--30680/fn--30681 (compiler.clj:696)#2023-05-0508:21Adrian SmithI think e/*http-request* has to be inside a e/server block
Is that an issue in your code base?#2023-05-0600:33VincentGood question! i have started again from another end of the cosmos and things seem to be working#2023-05-0600:34VincentI started with the ext chat demo and now added xtdb and have both. somehow going the other way was not working#2023-05-0600:34Vincent(xtdb-demo -> grab the cookie thang) was not worky dunno why but i did it the other way and it is going smoothly, just to be clear 😅#2023-05-0504:17VincentSo I did add that from the demo_extended_chat and I wonder if it's just something I gotta use that example for and work backwards from#2023-05-0509:37braai engineerWhy does http://Fly.io have to care about HYPERFIDDLE_ELECTRIC_SERVER_VERSION in prod? Do I need to hardcode this in fly.toml? Do I need to bump it every time I deploy?#2023-05-0509:50braai engineerI made a new project based on electric-app-starter (w/o cloning the repo) where I created each file one-by-one to understand what’s needed (and created a new app on http://Fly.io with my own app name). Am running into this error despite running all the commands in the starter README:
#19 90.44 Execution error (ExceptionInfo) at prod/fn (prod.clj:8).
#19 90.44 $HYPERFIDDLE_ELECTRIC_SERVER_VERSION must be set in prod
#19 90.44
#19 90.44 Full report at:
#19 90.44 /tmp/clojure-8647888368727244535.edn
#19 90.59 Execution error (ExceptionInfo) at clojure.tools.build.tasks.compile-clj/compile-clj (compile_clj.clj:112).
#19 90.59 Clojure compilation failed, working dir preserved: /tmp/compile-clj1089635137239597516
#19 90.59
#19 90.59 Full report at:
#19 90.59 /tmp/clojure-5380082418589440545.edn
------
Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'$VERSION'"']: exit code: 1
Am I supposed to add this env var to fly.toml? I added this to fly.toml and it still fails:
[env]
HYPERFIDDLE_ELECTRIC_SERVER_VERSION = "16f8e7b-dirty"
Does Electric Server Version refer to an Electric library version of my app’s version? If it’s my app version, what is the point of all the dockerization if it can’t use the hash “just deploy what’s there”? Sorry I am mere mortal with deployment PTSD from TeamCity days.#2023-05-0509:50braai engineerHow do I deploy this without GitHub Actions? I usually use CircleCI.#2023-05-0509:51braai engineerHad to add FLY_API_TOKEN to repo Secrets. I guess this is obvious if you’ve recently used http://Fly.io but I haven’t for a while.#2023-05-0510:02braai engineerWhere does HYPERFIDDLE_ELECTRIC_SERVER_VERSION come from? Looks like it is set to $VERSION. Where does that come from?#2023-05-0510:06braai engineerIt also fails locally if I run docker build --build-arg HYPERFIDDLE_ELECTRIC_SERVER_VERSION=$(git describe --tags --long --always --dirty) -t <my-tag-name> . and “Version: ” seems to be empty, but if I run f73b99a.
something about,
4.11.pom from central
=> => # fatal: not a git repository (or any of the parent directories): .git
=> => # Cleaning up
`
(this folder is a git repo)#2023-05-0510:07Geoffrey GaillardHYPERFIDDLE_ELECTRIC_SERVER_VERSION and its sibling HYPERFIDDLE_ELECTRIC_CLIENT_VERSION ensures your app clients and server versions stays in sync across deployments.
You can look at the Dockerfile. The CMD line sets HYPERFIDDLE_ELECTRIC_SERVER_VERSION .
You do not need to add it to fly.toml , it’s already baked in the container image produced by docker build.#2023-05-0510:17braai engineerHi @U2DART3HA. I see in Dockerfile it is being assigned from whatever $VERSION is, which seems to be a build-arg thing (new Docker feature I guess).
When I run these commands locally, it builds everything then fails at docker build because the version is empty:
clojure -X:build build-client # optimized release build
clojure -X:build uberjar # contains demos and demo server, currently
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty`
docker build --build-arg VERSION='"'$HYPERFIDDLE_ELECTRIC_APP_VERSION'"' -t electric-starter-app .
docker run --rm -p 7070:8080 electric-starter-app
Error:
#20 99.11 Execution error (ExceptionInfo) at prod/fn (prod.clj:8).
#20 99.11 $HYPERFIDDLE_ELECTRIC_SERVER_VERSION must be set in prod
#20 99.11
#20 99.11 Full report at:
#20 99.11 /tmp/clojure-2398244998172268921.edn
#20 99.26 Execution error (ExceptionInfo) at clojure.tools.build.tasks.compile-clj/compile-clj (compile_clj.clj:112).
#20 99.26 Clojure compilation failed, working dir preserved: /tmp/compile-clj14120230338945615196
#20 99.26
#20 99.26 Full report at:
#20 99.26 /tmp/clojure-4967785673739274943.edn
Did I miss setting a local env var? I even tried running docker build with VERSION set manually and it still fails: docker build --build-arg VERSION='"d46d452"' -t electric-starter-app. I can see that Version: is unset Building client. Version:
and #20 83.70 Compiling server. Version:
These vars are set on the shell I am running docker build on my host machine:
electric-blog git:(main) ✗ echo $HYPERFIDDLE_ELECTRIC_APP_VERSION
d46d452
➜ electric-blog git:(main) ✗ echo $HYPERFIDDLE_ELECTRIC_SERVER_VERSION
d46d452
#2023-05-0510:28braai engineerMore debugging: if I explicitly set the variable with an ENV directive in the Dockerfile, it seems to pick it up but the build still fails (I’m guessing because separate build contexes):
...
ENV VERSION="d46d452"
RUN clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'$VERSION'"'
=>
=> [build 9/9] RUN clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'d46d452'"' 108.2s
...
#22 167.9 Bundling sources
#22 168.1 Compiling server. Version: d46d452
#22 182.6 10:21:56,785 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
Fail:
#22 182.6 10:21:56,926 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@75a37ebb - Registering current configuration as safe fallback point
#22 182.6
#22 185.0 Execution error (ExceptionInfo) at prod/fn (prod.clj:8).
#22 185.0 $HYPERFIDDLE_ELECTRIC_SERVER_VERSION must be set in prod
#22 185.0
#22 185.0 Full report at:
#22 185.0 /tmp/clojure-10111137647951511898.edn
Now updating my Docker Desktop version to see if that helps.#2023-05-0512:28braai engineerTo reproduce with Docker Desktop v4.13.1 and Engine v20.10.20:
git clone
Error:
[:prod] Build completed. (153 files, 1 compiled, 0 warnings, 55.44s)
Bundling sources
Compiling server. Version: 0fa1b26
12:45:44,605 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
12:45:44,606 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/petrus/Projects/electric-starter-app/target/classes/logback.xml]
...
12:45:44,738 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@fd69679b - Registering current configuration as safe fallback point
Execution error (ExceptionInfo) at prod/fn (prod.clj:8).
$HYPERFIDDLE_ELECTRIC_SERVER_VERSION must be set in prod
#2023-05-0512:35Geoffrey GaillardThank you for this repro, I’ll give it a look 👍#2023-05-0513:08braai engineerUpdated my Docker Desktop for macOS to latest version - same issue.#2023-05-0514:27Geoffrey GaillardThere was some issues, sorry for the confusion.
I pushed some fixes. You can try with the latest Readme.
This should work:
git clone
#2023-05-0515:07braai engineer(hold up might be env override breaking it)
~fly deploy completes now but app fails at runtime:~
2023-05-05T15:06:24.983 app[d2a5ba3c] jnb [info] Starting init (commit: 083df78)...
2023-05-05T15:06:25.030 app[d2a5ba3c] jnb [info] Preparing to run: `/bin/sh -c java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$VERSION -jar app.jar` as root
2023-05-05T15:06:25.051 app[d2a5ba3c] jnb [info] 2023/05/05 15:06:25 listening on [fdaa:0:6591:a7b:d5a6:d2a5:ba3c:2]:22 (DNS: [fdaa::3]:53)
2023-05-05T15:06:30.985 health[d2a5ba3c] jnb [error] Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.
2023-05-05T15:06:33.494 app[d2a5ba3c] jnb [info] Exception in thread "main" clojure.lang.ExceptionInfo: HYPERFIDDLE_ELECTRIC_SERVER_VERSION jvm property must be set in prod {}#2023-05-0515:27braai engineerok nice works now 👍 thx @U2DART3HA#2023-05-0601:12Dustin Getzto be clear @U051SPP9Z this env var is used to control auto refresh of stale client app when the server deploys. Which you need otherwise your stale tabs will reconnect to a mismatched server version which will result in spooky undefined behavior (mostly just errors)#2023-05-0511:00Yab MasI'm running into some issues while trying to interop with react.
I basically followed the example in the docs but run into the following error: Can't take value of macro blabla/with-reagent when trying to boot.
To make sure I wasn't doing anything wrong in my react -> reagent interop I copied your demo and tried to boot that but now Im getting the warning: Use of undeclared Var blabla/with-reagent and if I load the page anyway I get Cannot read properties of undefined (reading 'cljs$lang$applyTo') in ( blabla/with-reagent #object[blabla$interop_demo$TinyLineChart] in the browser console
I really just copied the demo-file https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_reagent_interop.cljc and tried to boot it with:
(defn ^:export ^:dev/after-load init! []
(current-app/initialize-app
(e/boot (binding [dom/node js/document.body]
(interop/ReagentInterop.)))))
I'm on com.hyperfiddle/electric {_:mvn/version_ "v2-alpha-263-g89da9d11"}and obviously added the npm recharts dep.
I don't understand enough of the internals of the demo_reagent_interop file to know how to proceed with debugging, any tips?#2023-05-0511:04xificurCcan't take value of macro usually means you have a cljc file with macros but didn't do #?(:cljs (:require-macros my-ns))#2023-05-0511:33Yab MasThat was it. Should have been able to figure that one out... A bit overwhelmed by not fully understanding so many parts, haha. Wondering why it's missing from the demo_reagent_interop.cljc then though? Anyway, thanks for the quick response!#2023-05-0511:36Yab MasAh, nvm, I now see it's present in the demo on the examples-website. I was looking in the repo. Guess those are out of date? Will look on the website from now on then.#2023-05-0511:38Dustin Getzwhich file is out of date?#2023-05-0511:49Yab MasI'm not sure if it's out of date but I'm not seeing the require-macros line in https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_reagent_interop.cljc. It's probably in another file that I missed.#2023-05-0513:14Dustin Getzyeah that looks busted#2023-05-0513:14Dustin Getzthat demo isn't enabled in that repo, it got copy pasted to the tutorial and fixed in flight likely#2023-05-0513:15Dustin Getzthe demos in the repo master only exist for us at this point, because master is getting some experimental stuff as we rework our form controls, it's all unreleased, nobody should be running master at this point unless you are talking to us frequently#2023-05-0513:19Yab MasCool, good to know, thanks#2023-05-0513:10Yab MasHow would I go about triggering an electric-function from a react-component? I looked at the demo's but don't believe it's in there. What I'm looking for is something like an onClick in the reagent component and than running a function where I can do (e/server (some-action))#2023-05-0513:11markaddlemanI’ve been struggling mightily with that myself.#2023-05-0513:12Dustin Getzthe quick and dirty way is to use an atom#2023-05-0513:12Dustin Getztry that and then send me the whole integration as a gist and i will come up with a better integration#2023-05-0513:27markaddlemanThis is a simple and somewhat tongue-in-cheek example but it accurately represents the general problem. I have a third-party component that sets up divs to which I want to attach electric elements. Of course, the divs are not available until after the component is mounted on the dom. The component allows me to pass a callback to be invoked after the mount occurs.
The problem is that the callback occurs outside of the electric context. Furthermore, it’s not clear how an atom would solve this problem.#2023-05-0513:27markaddleman(hit enter a little too soon)#2023-05-0513:29markaddlemanNow that I type this out, I think I see a solution to using an atom to solve this.#2023-05-0513:30markaddlemanI’ll explore and post back#2023-05-0513:32Dustin GetzUnderstood, to answer your proposal directly (I sidestepped it earlier this week), the thing you want (call Electric from Clojure) violates the function color problem#2023-05-0513:32Dustin Getzhttp://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/#2023-05-0513:32Dustin GetzElectric must be on the outside#2023-05-0513:36markaddlemanI’m curious why we couldn’t declare the color and pass some electric environment as part of the callback fn. To my (very very) naive eyes, this looks like a subset of the boot problem.#2023-05-0513:36Dustin Getzwhat's "boot problem"#2023-05-0513:37markaddlemanhyperfiddle.electric/boot#2023-05-0513:37markaddlemanIt invokes an EFn from a vanilla javascript context#2023-05-0513:45Dustin Getz#2023-05-0513:47Dustin Getzm/observe is the resource management primitive with object lifecycle, the resource here being allocating a callback (that you must remember to clean up)#2023-05-0513:51markaddlemanThis solution uses the event system to route control into an electric context. If I understand correctly, the atom solution is a similar pattern.#2023-05-0513:57Dustin Getzyes, m/observe is the idiomatic way (and forces you to write the cleanup code). it uses a side effect emit! to bridge the javascript callback into an event stream and contain that side effect inside an encapsulated box. The atom quick-n-dirty approach uses a similar side effect, instead of emit! you write reset! or swap! and you end up with (let [click (e/watch !x)]...) - but minus the destructor code. and also it doesn't force you to consider the m/reductions call which is essential complexity, which forces you to think about the continuous/discrete boundary (i.e. backpressure)#2023-05-0514:00markaddlemanGotcha. All that makes sense. In my dashboard component example, the library essentially requires a vanilla callback fn from which I want to execute (is “signal” the better word?) some electric fn. I think an atom is a more straightforward approach compared to m/observe et al.#2023-05-0514:07Dustin Getzok, if it works for you then go for it, the devil is in the details (as i said, the discrete/continuous boundary is essential complexity, so hacking it with an atom is making some assumptions)#2023-05-0514:08markaddlemanOne more bit of that complexity: The dashboard component has an imperative crud-like interface: addPanel, updatePanel, removePanel.#2023-05-0514:10markaddlemanI can write my own diff-ing logic but it occurs to me that e/for must do something similar. Is electric’s diffing logic available for clients?#2023-05-0514:46Yab MasThanks a lot for your input both, much appreciated. I'll need some time to digest but I can see this all making sense to me with a bit more effort, just a bit drained for today. I haven't got it fully working yet, but i'll get back to it soon. Two things arent fully clear to me atm. The dom/node that the eventListener targets is the one from the binding-form in e/boot right? If I want to target a different dom-node, should I do another binding-form around this addEventListener call? This is what Im trying but havent succeeded yet. The other thing that confuses me for now is that I only see the result of the first click, I put a prn inside the (when click...) form and only see it once. I also placed a prn inside the emit-fn and see this one is called on every click. Is this the expected behaviour? I'll look at the missionary-docs a bit more later, maybe I can figure it out then. Anyway, thank again for the input, i'll get back to this soon!#2023-05-0515:22Dustin Getzthe dom/node is the "current" dom node "here" as maintained by electric-dom#2023-05-0515:23Dustin Getztry (dom/h1 (println dom/node)); dom/node will be h1#2023-05-0515:24Dustin GetzAs to your questions about clicking, please post the full code in a gist#2023-05-0515:25Dustin GetzIf you are writing (when x (prn "hello")), (prn "hello") will only run when its arguments change, and both "hello" and `prn are constants, so the expression will run once even if x rotates through several truthy values in a row#2023-05-0515:27Dustin Getz(when x (prn x)) -- will have more interesting behavior because the prn depends on x#2023-05-0515:29Dustin Getz@U2845S9KL i dont understand the leap from "imperative component interface" to "writing diffing logic"#2023-05-0515:29Dustin Getzare you trying to build a declarative Electric-native syntax that is backed by the javascript library?#2023-05-0601:21markaddlemanYes, that’s exactly what I’m doing. Now that I understand that interop is through the missionary event/signal system using atoms or an event queue, it’s going pretty well. I should have the first fully wrapped component tomorrow#2023-05-0817:44Yab MasSo I worked on this again today, made some progress, but don't feel like I fully comprehend it yet.
At a high-level I understand the situation: the link with the function-color article and using the js-eventsystem with the missionary primitives to make the bridge.
My misunderstanding of dom/node and when have been corrected by your comments and a bit of experimentation. The fact that in the when-body only forms with changed arguments run makes me doubt if it's really well suited for what Im trying to do: i.e. call a function that does some stuff on the backend with help of e/server. I've got this working, but I can still only click the button once and have the function running. Consecutive clicks don't change the values with which it's called so it's not running. It's not clear to me how (if?) I can use this structure to have a click in the reagent component result in the trigger of a electric-fn (without changed args) 1-on-1.
For the first thing that I wanted to do, catching the click-event was enough. But then I needed to capture some form-data aswell, i.e. use data from the reagent-form in a electric-fn-call with e/server. I guess I need to make a custom-js-event with the added data for this and capture that via the same structure? I haven't got this working yet but that might also be because I've no experience working with js-events like this in js/cljs either so maybe something goes wrong there, will work on it a bit more.
In general it feels like Im working against the grain of electric here. Im totally willing to put in some more effort to get this to work, and I could see that it's possible to hide this interop via the js-event-system behind some abstraction once I understand it and be back to less verbose syntax. But it's not clear to me if it's just grasping this bit or if stuff like this will keep popping up by trying to integrate react and electric like this.
We use react because the platforms we integrate with provide styled-components in it. This is super convenient and something people won't be willing to give up quickly, but I'm tempted to see how things workout if I rebuild the components with native electric-structures and do the styling myself. What your point of view on this?#2023-05-0818:49Dustin GetzSend me your complete code and I will take a look. I can't tell specifically what you are trying to do so it's hard to comment#2023-05-0818:52Dustin GetzWe don't have any paid customer projects today running heavy react interop so we have not invested much time thinking about it other than to just demonstrate that it is possible to do. Once we understand what kind of things people are trying to integrate, we can provide idioms and documentation, and build any necessary helpers#2023-05-0914:06Yab MasSo what I'm trying now is a combination of:#2023-05-0914:07Yab Mas[:div
{:id "create-item-link"
:onClick #(->
(.getElementById js/document "create-item-link")
(.dispatchEvent (js/CustomEvent. "react-event"
(cljs.core/clj->js {:bubbles true
:detail {:action "create-item-link"
:data @selected }}))))}
"Submit"]]#2023-05-0914:07Yab Mascustom events being triggered inside reagent-component#2023-05-0914:08Yab Mas(let [react-events (->> (m/observe (fn mount [emit!]
(let [f (fn [e] (emit! e))]
(.addEventListener dom/node "react-event" f)
(fn unmount []
(.removeEventListener dom/node "react-event" f)))))
(m/reductions {} nil)
(new))]
(when react-events
(let [event (js->clj (.-detail react-events) :keywordize-keys true)]
(case (:action event)
"create-item-link" (let [target-item (->> (get-in event [:data :monday-item-id]) (hash-map :item/id))]
(e/server
(new db-proxy/create-item-link selected-item target-item)))))))#2023-05-0914:08Yab Mascode listening to this inside electric-component#2023-05-0914:08Yab Mas(e/defn create-item-link [source-item target-item]
(e/server
(let [item-link (-> (item-links/create-item-link source-item target-item))]
(reset! !item-link item-link))))#2023-05-0914:11Yab MasIs the db-proxy/create-item-link call you see in the second snippet. It's just a hacky way to deal with the fact our db doesnt fit the electric-model so I do the db action and then alter the atom that the components are watching. It feels a bit weird having to call it with new as its really intended as a function-call here and not as the initialisation of an electric-component but I havent yet figured out another way to dot this#2023-05-0914:16Yab MasThis kind of works except the issue I mentioned before that the functions only gets triggered once. Which makes sense given my understanding of electric but doesn't give me the behaviour Im looking for, i.e. being able to trigger the action on every click.#2023-05-0914:20Yab MasI don't mind sharing the complete code but its really just me experimenting a lot atm, so I thought this would be easier for you#2023-05-0915:42xificurCcan you bind the onclick handler in electric code?
(dom/on (dom/by-id "create-item-link") "click" (e/fn [e] ...))
#2023-05-0915:54xificurCreading this thread, @U2845S9KL your initial example could be solved the same way, right? You wrote
> The component allows me to pass a callback to be invoked after the mount occurs
You can (reset! !my-div thisDiv) in there and have in electric
(let [my-div (e/watch !my-div)]
(when my-div
(dom/on my-div "click" (e/fn [e] ...)))#2023-05-0916:58Yab MasEuhm, haven't tried that before, seems a lot less complicated than the missionary-event route, but didnt get it to work immediately.
Just to make sure I've got the correct idea: we just render the react component like in the interop example but add id's for the elements we want to catch the clicks from. Than in our elecric-code after we mount the react-component via the with-reagent macro, we do (dom/on (dom/by-id "the-id-we-added") "click" ...)
I'm having trouble binding this because the dom-node doesn't seem to exit yet when we try to bind the click-handler. The code in the following snippet prints nil for the html-element:
(dom/div
(dom/style {:color "#FFFFFF"
:display "flex"
:padding "5px"
:justify-content "flex-end"})
(interop/with-reagent DropdownMenuComp item-link)
(prn (str "html-element: " (dom/by-id "delete-item-link")))
(dom/on
(dom/by-id "delete-item-link") "click"
(e/fn [e] (e/server (prn "deleting on server"))))#2023-05-0917:04Yab MasOk, nvm these last comments, these were a bit silly. I think I understand the proposed structure now, will try to work it out.#2023-05-0918:54Yab MasYeah it's working! What I ended up doing is emitting the events like I did in what I posted before but now from the component-did-mount call-back instead of the on-click of the reagent-component. Then in the handling of the events in the electric-environment I did the (dom/on (dom/by-id "x") "click" .... ) And I now have that function being run on every click, great! I still have some cleaning up todo and will see if I can find the time to write some helpers that hide thi stuff, but happy to have it working to begin with 🙂#2023-05-0918:56xificurCGlad to hear. You can use the pattern I sent to Mark above - reset! your div into an atom in component-did-mount and use the snippet#2023-05-0611:30Jhttps://twitter.com/vinchbat/status/1654361228997787648?s=46&t=FLgxpB8u-zgNc4wlhiD7Gw#2023-05-0617:57Vincentlol back to PHP#2023-05-0617:57VincentSQL into the HTML is like whoa#2023-05-0618:41JI’m curious how this works internally! #2023-05-0621:15lilactownthe 'use server' directive causes the compiler to split it into a function that gets run on the backend, and a function that on the client will send an RPC call to the backend function#2023-05-0614:52Adrian SmithJavaScript has a ResizeObserver for listening to resize events for individual dom elements
How would you integrate it into the life cycle of electric?
https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver
https://stackoverflow.com/a/39312522#2023-05-0614:56Dustin Getzm/observe#2023-05-0614:57Dustin Getzsee https://clojurians.slack.com/archives/C7Q9GSHFV/p1683292217371419#2023-05-0615:03Dustin Getzalso see this IntersectionObserver snippet: https://clojurians.slack.com/archives/C7Q9GSHFV/p1669672518189879?thread_ts=1669672330.176349&cid=C7Q9GSHFV#2023-05-0917:19Adrian SmithThis is where I have got to:
I'm not sure how to feed my custom listener the dom elements as created by dom/div in my Meadow component#2023-05-0917:24Adrian Smithhmm doing some searching in channel, maybe using dom/node inside the context of dom/div will work let's see#2023-05-0917:27Dustin Getzplease can you paste the code as a single slack attachment - its too hard to read like this due to slack's shit UI#2023-05-0917:28Dustin Getz(don't use backtick, use cmd-shift-enter)#2023-05-0917:33Adrian Smith#2023-05-0917:34Adrian SmithI think it's struggling with js/ResizeObserver. at the moment#2023-05-0917:35Dustin Getz"its struggling" ?#2023-05-0917:36Adrian SmithOn the backend there's a stack trace that has "Caused by: java.lang.RuntimeException: No such namespace: js"#2023-05-0917:36Dustin Getzyou may need to guard the whole function to be cljs only#2023-05-0917:36Dustin Getzthat function doesn't compile as clojure#2023-05-0917:37Dustin Getzalso, L19 - illegal e/server inside a clojure lambda. electric stuff is only valid in electric fns#2023-05-0917:38Adrian SmithYeah I switched back to defn, when was trying something, I've put it back to e/defn, I'm conscious I'm trying to return a m/observe object#2023-05-0917:43Dustin Getzlisten-for-resize should be a clojure function, it doesn't contain any reactive code#2023-05-0917:43Dustin GetzMissionary is a clojure/script library, not an electric library#2023-05-0917:44Dustin Getzchange (emit!) to (emit! (.-contentRect entry))#2023-05-0917:46Dustin Getzthe flow returned by listen-for-resize needs an initial value, you can add nil with m/reductions, or call (emit! nil) once right away at the beginning of mount. Then, bridge to electric with new :
(dom/div (new (listen-for-resize dom/node)) ...)
#2023-05-0917:47Dustin Getzmove the e/server stuff to the electric body
(dom/div (let [{:keys [width height]} (new (listen-for-resize dom/node))]
(e/server (swap! !nodes merge (.-id entry) {:width width :height height}))
...))
#2023-05-0919:33xificurCHere's a minimal interop wrapper that you call as (new (size> my-dom-node)) and returns a reactive view of [width height]. Change based on your needs.#2023-05-0920:34Adrian SmithAh thank you, I need to clean up my code but it works! 😄#2023-05-0618:01Craig BrozefskyMet @dustingetz at the conj and he told me about electric clojure, just wanted to stop by after I had a chance to look at it and say Nice Work!#2023-05-0618:02VincentOh good he's gettin' the word out! It's the future 😅#2023-05-0618:03Vincent😁 have you had a chance to tinker with it yet? it's a little bit of a mind wrap but really satisfying#2023-05-0618:03Craig Brozefskyjust the live tutorial. I'll take a closer look when I have a UI to build for some internal tooling#2023-05-0618:53Vincentnice. yeah the live tutorial explains a ton.#2023-05-0717:36Alan CarrollI'm trying to recreate the 7GUIs Counter example in portfolio (the idea being ultimately you would be able to switch between electric components similar to the tutorials pages, just in the portfolio UI instead).
Everything renders as I would expect it to, but clicking the button doesn't do anything.
(e/def Scene (e/client (e/watch misc/!Scene)))
(e/defn App []
(portfolio.ui/start!)
(e/client
(let [!state (atom 0)]
(binding [dom/node (:div Scene)]
(dom/div
(dom/p (dom/text (e/watch !state)))
(ui/button (e/fn [] (swap! !state inc)) (dom/text "Count")))))))
Changing the !Scene atom through the portfolio UI correctly updates :div and dom/node, if I add js/console.log statements under the binding form and print them out, they run whenever !Scene changes, as I would expect. It's just the button on the page doesn't do anything! It may be worth noting that the UI loads the div's into an iframe. Even though they render fine, could that be cutting off electric reactivity?#2023-05-0718:11Dustin Getztry println, the code looks fine#2023-05-0718:14Dustin Getzsend me a link to “Portfolio” docs i’ll take a quick look#2023-05-0718:23Alan Carrollhttps://github.com/cjohansen/portfolio/tree/main#Usage#2023-05-0718:38Alan CarrollLike this? (added when-let just to silence a null console error on refresh)
(e/client
(let [!state (atom 0)
state (e/watch !state)]
(when-let [d (:div Scene)]
(binding [dom/node d]
(println state)
(println d)
(dom/div
(dom/p (dom/text (e/watch !state)))
(ui/button (fn [] (swap! !state inc)) (dom/text "Count")))))))
state and d both print (0 and div element) when the scene loads, but clicking the button doesn't do anything more. If I navigate to another default scene and back, !Scene is reset with a new div to bind to dom/node, and (println d) runs and prints the div element to console, but (println state) only runs once on refresh.#2023-05-0719:15Alan CarrollI've also tried adding :state to my !Scene atom and updating that directly -- assume that since it's reacting to changes in :div then if the button worked at all it should also react to changes in :state , but no luck.
(e/client
(let [d (:div Scene)
state (:state Scene)]
(binding [dom/node d]
(println state)
(println d)
(dom/div
(dom/p (dom/text state))
(ui/button (e/fn [] (swap! misc/!Scene update-in [:state] inc)) (dom/text "Count"))))))
I've tried e/fn and fn for the button, not sure if that makes any difference here, but neither work anyway.#2023-05-0719:58Dustin GetzCan you send me a repo to clone and look at?#2023-05-0804:59Alan Carrollhttps://github.com/alan-carroll/electric-portfolio#2023-05-0723:29tlonisthttps://github.com/green-labs/electric-projects
greenlabs' first electric project is already serving 900k customers and is made public!
It is a simple attendance checking app with
• authentication using access/refresh token
• calendar view
• simple db manipulation
99% of the work was done by @orange881217, nice job done!#2023-05-0800:11Dustin Getzwait what, please clarify “serving 900k customers”?#2023-05-0800:15tlonistah, there can be a misunderstanding.
The electric app was inserted as a webview to our existing react-native application, which has 900k users.
Attendance feature was originally outsourced to other Saas solution in Korea, and has been a very popular feature for users.#2023-05-0800:16Dustin Getzaha got it#2023-05-0800:16Dustin Getzi read the code, nice job!#2023-05-0806:20moocarThanks for posting @U01TFN2113P! I've been trying to build an electric app and hoping someone would post a larger project to compare against.#2023-05-0814:26Dustin GetzWhat is the expected load in simultaneous connected users for this app?#2023-05-0814:26Dustin GetzHas it been stress tested yet?#2023-05-0822:50tlonist> Has it been stress tested yet?
No it's not tested yet. Users typically react to push notifications and that's around 10~20k per minute#2023-05-0800:08VincentWow super cool! @tlonist.sang i'm really glad it's up so i can read through the source and learn more. Looks neat, is it for "confirming" attendance to events?#2023-05-0800:16tlonistyes it is. What it does is
• user enters and sees the marks on calendar on the days he/she attended
• user marks today as attended
• all these with authenticating tokens#2023-05-0804:11NikolaiWant to share the prototype of implementing the Croquet VM in Electric - Clojure. https://github.com/NikolaySuslov/krestianstvo-electric
In Croquet application architecture the state is never going outside the client and is updating in sync by using local internal queues of messages (Future message sends). These queues advanced in virtual time by receiving the control tick (metronome) message from the Reflector. All user-land events are also stamped by Reflector and distributed to all the clients back (sending owner too), forming very low net traffic. I am interested in how this suits Electric, and if this type of architecture can be realised even without introducing Croquet on top of it?#2023-05-0812:35Dustin GetzThis is super cool, and impressive!
Extracting some links while I read it:
https://www.krestianstvo.org/
https://docs.krestianstvo.org/en/introduction/
https://en.wikipedia.org/wiki/Croquet_Project
https://www.krestianstvo.org/docs/about/publications/
https://blog.codefrau.net/2021/08/what-is-croquet-anyways.html
; Reference implementations:
; Virtual Time
; Reflector server
; Future Message Send ()
; TODO: Portals
#2023-05-0812:39Dustin Getz> I am interested in how this suits Electric, and if this type of architecture can be realised even without introducing Croquet on top of it
• Electric runtime is backpressured, which means that if you pause the network (try it with browser dev tools), messages are queued and will go through when network is unpaused. We see "offline as just online with extreme latency". This builtin capability seems to overlap a lot with this userland event system
• The mouse cursor is lagged presumably because it is round tripping through the server? We're working on improving our optimistic state abstractions for cases like this (we are focused on perfecting our crud forms first, but the underlying primitives should be the same for things like mouse cursors)#2023-05-0812:46Dustin GetzI don't really understand the Croquet architecture yet, specifically I don't understand this exactly yet:
> In Croquet application architecture the state is never going outside the client and is updating in sync by using local internal queues of messages (Future message sends). These queues advanced in virtual time by receiving the control tick (metronome) message from the Reflector. All user-land events are also stamped by Reflector and distributed to all the clients back (sending owner too), forming very low net traffic.#2023-05-0812:49NikolaiYes, there is a good explanation of Croquet VM on their current version (Croquet OS) https://croquet.io/croquet-os/#2023-05-0812:51NikolaiCroquet OS is closed sourced, but it has the same architectural components, that I try to implement currently (based on Virtual World Framework aka Croquet IV)#2023-05-0812:51Dustin GetzWhat is your relation to this project? Krevistiansvo is your project?#2023-05-0812:52Dustin Getz"Croquet OS provides a kernel that enables bit identical shared simulations between users."
Is there a rationale for this capability? Why is bit-identical simulation something that virtual worlds need#2023-05-0812:54NikolaiYes, I am a founder and developer of Krestianstvo project. I am working with Croquet since Smalltalk version.#2023-05-0812:55NikolaiSure, bit-identical simulation is a crucial concept for virtual worlds, as when simulation is going no network traffic is involved. Clients getting only timestamps from reflector, which advanced the simulation.#2023-05-0812:56Dustin Getz"when simulation is going no network traffic is involved" -> what about syncing user I/O?#2023-05-0812:59NikolaiAll user-land events needs to go to Reflector to be stamped and then distributed to all the clients back. That's considered as the minimal traffic (like mouse coordinates or clicks). The application state is stying on client side.#2023-05-0813:01Dustin Getzhow is latency mitigated?#2023-05-0813:03Dustin GetzAh, I guess the simulation state excludes cameras for example, it's the scene graph which is synced and consistent?#2023-05-0813:06NikolaiRight, users operate through local Views on a synced Model.#2023-05-0813:08Dustin Getzif the model state is a reduction over global event log, and this reduction is replicated on each client (peer?), then the global event log is replicated to each client/peer, so sorta like a bitcoin node? What if the number of events is large#2023-05-0813:09Dustin Getzreading this https://blog.codefrau.net/2021/08/what-is-croquet-anyways.html#2023-05-0813:12NikolaiYes, that's great introduction! As for the latency, clients with low speed network will not pause the clients with hight speed traffic, that's was the key feature of Croquet. In the contrary to lockstep algorithms.#2023-05-0813:14Dustin GetzSo the reflector runs in the edge, all events are funneled through the reflector (which is like a transactor, it decides a globally consistent total ordering). It sounds like https://www.cloudflare.com/products/durable-objects/#2023-05-0813:17Dustin GetzTo map to electric/missionary:
synced Model - this is discrete event streams, in simplest webapp terms this is basically the database transaction log?
local Views - this is continuous time lazy rendering, electric maps best to this part#2023-05-0813:18Dustin GetzThe question of if the synced model (the transaction log) is replicated or centralized - Electric doesn't care, we would say this is a data plane decision, some databases replicate and others are centralized#2023-05-0813:20Dustin GetzThe Croquet event layer is probably better than ours, this is the part where there is overlap, it is comparable to our optimistic crud form sync abstractions which we haven't really talked about or shown yet. Our optimistic form state sync is done in Electric userland not the runtime#2023-05-0813:20Dustin GetzSo it makes sense that there is overlap, and your project makes sense in an Electric worldview#2023-05-0813:20Dustin GetzDid I get this right?#2023-05-0813:22Dustin Getzgreat#2023-05-0813:22Dustin GetzHave i answered your original q?#2023-05-0813:24NikolaiYes, thank you Dustin! I am really excited with Electric and would like to do a lot with it!#2023-05-0813:50NikolaiThat's demo video of the prototype.#2023-05-0814:01Dustin GetzIs this accurate:
The big idea is to use the Electric distributed runtime as a basis for a higher level scene graph sync abstraction for use in collaborative virtual worlds.#2023-05-0901:49NikolaiHere is a good explanation video of David P. Reed at OOPSLA '05 talking about Croquet architecture.
"Designing croquet's TeaTime: a real-time, temporal environment for active object cooperation" https://dl.acm.org/doi/10.1145/1094855.1094861#2023-05-0914:54NikolaiHere are the slides from the presentation.#2023-05-0811:22Aziz Aldawood@dustingetz what are your thoughts on next.js server actions?#2023-05-0812:43Dustin GetzI think (last i looked) server actions are RPC (meaning not a scalable composition model), tbh I struggle to manifest the attention to slog through all the complexity and adhoc machinery to figure out all the ways their thing is broken, perhaps someone who has used it can compare#2023-05-0816:15braai engineerAnyone need help at their company porting any apps to Electric for the performance benefits? I am looking for project work and would love to spend more time working with Electric ⚡.#2023-05-0821:01Erich OceanHas anyone integrated a standard 3rd party React component with Electric? I'm looking for sample code.#2023-05-0821:15xificurCWe have a https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_reagent_interop.cljc. Does that help?#2023-05-0821:20Erich OceanYup, that's enough. Thanks!#2023-05-0823:34Erich OceanWhy is new required here? https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L33#2023-05-0823:35Erich OceanIt seems like new is being called on the result of the e/fn call (macro?) here: https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L38-L41#2023-05-0823:36Erich OceanSpecifically, why not just (F v) on line 33?#2023-05-0823:59nooniannew is used to call electric functions. Since F and InputSubmit are electric functions you have to call them with new. Note that the e/defn macro will rewrite the code so it has the semantics of a function call and is not a regular usage of new#2023-05-0900:04Dustin Getzhttps://electric.hyperfiddle.net/user.tutorial-lifecycle!%4Cifecycle#2023-05-0900:06Erich OceanGot it, thanks.#2023-05-0823:51Vincentother than e/for-by are there any other constructs that iterate over collections? I have a simple query return result sets and i want to draw them, but not sure how to e/for-by it.#2023-05-0900:05Dustin Getzjust e/for-by, should work with anything sequable. https://electric.hyperfiddle.net/user.demo-system-properties!%53ystem%50roperties#2023-05-0900:27VincentI get a #{set} of [vecs of {maps}] so it's interesting figuring out how to unpack it#2023-05-0900:28Vincent#{[{:details "deets'}] [{:details "detes"}]}
not sure how to `e/for-by it#2023-05-0900:28Vincentideally the input to e/for-by is a vec of maps?#2023-05-0900:42VincentI got it working but it looks mental 😅
tracks (-> (xt/q db '{:find [(pull ?track [:track/id :track/album-id :track/title])]
:where [[?track :track/album-id album-id]]
:in [album-id]}
album-id)
vec
flatten
vec)]
vec flatten vec :face palm:#2023-05-0900:44Vincentand then for whatever reason calling it just track works instead of track/title
(dom/div (dom/props {:class "track-cool"})
(e/server
(e/for-by :track/id [{:keys [track/id track/album-id track/title]} tracks]
(e/client
(dom/div (dom/text (str "track " title)))))))
i think that's by design?#2023-05-0908:51braai engineer@U055PQH9R4M the flatten will be expensive. You can do (map first (x/q db …)).
@dustingetz is it advisable to pass a sorted seq to e/for-by or does the keyfn imply a sorting seq? I assume you’d want a sorted result but also keyed for efficient updates.#2023-05-0910:33Dustin Getze/for-by does not sort, it is a sequential traversal same as clojure for#2023-05-0910:35Dustin Getze/for-by performa diffing internally for stabilized reactive updates, which is a traversal, so pass reasonable collections (like a page of database records, not 1000 records)#2023-05-0910:36Dustin Getzthe DOM itself can only handle about 2k nodes anyway #2023-05-0911:19braai engineer@U055PQH9R4M I think you’ll want to add a :sort-by clause to your XT query, or sort-by the query result before passing to e/for-by to avoid the diffing, or does the diffing always occur?#2023-05-0912:01xificurCdiffing is necessary to handle updates effectively. It's the same as in e.g. react where you have a dynamically updating collection and need to provide a key#2023-05-0913:13Dustin Getzto be clear, diffing is separate from sorting. e/for-by renders dom elements in order, it can render any sequence not just UIs that are in sorted order#2023-05-0901:56Erich OceanI'm trying https://electric.hyperfiddle.net/user.demo-webview!Webview by adapting it to the electric-starter-app repo. All seven previous examples worked out-of-the-box, but this one is throwing an error:#2023-05-0901:56Erich Ocean------ WARNING #1 - :undeclared-var --------------------------------------------
File: /Users/erichocean/Desktop/xygroup/electric-starter-app/src/user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Use of undeclared Var app.todo-list/conn
--------------------------------------------------------------------------------
9 | (binding [hyperfiddle.electric-dom2/node js/document.body]
10 | (app.todo-list/Webview.))))
11 |
12 | (defonce reactor nil)
--------------------------------------------------------------------------------
#2023-05-0901:57Erich OceanHere's are the relevant files I've changed, along with (unchanged) deps.edn: https://gist.github.com/erichocean/8c61a27a3ac3f6b39061c7d4054e748c#2023-05-0902:56Vincentlooks pretty sane to me.#2023-05-0902:56Vincentmaybe change the defonce to def and see if the problem persists#2023-05-0905:38noonianconn is only defined in clj, therefore it will only exist on the server and I believe WebView is being called from a client context. You probably need to wrap the code referencing conn in an e/server block or call WebView from an e/server block#2023-05-0908:26Erich OceanI did try that and the error went away, but the content was still missing. I'm now running it from the Electric main repository, and it does work now. (Ironically, there's now an unrelated :underclar-var warning in the compilation log. This is using top-of-tree on the main branch.)#2023-05-0910:16Dustin Getzthe demos in master are unstable currently while we land a new set of changes, please do not use master#2023-05-0910:17Dustin Getzyou can clone the examples app, see our github#2023-05-0901:58Erich OceanDoes this example only work on a newer version of Electric?#2023-05-0908:29Erich OceanI'm playing with styling the Java system properties test page to test out styling with CSS. I've got my CSS file set as an absolute path in index.html (I'm editing the main Electric code). When I add a class and save, TailwindCSS updates the CSS in about 30ms, but Electric is taking 4-5 seconds to compile the code whenever I edit (dom/props {:class "tailwind-css classes go-here"}) and save.#2023-05-0908:30Erich OceanThe entire file is just 42 lines of code, so that's…concerning.#2023-05-0908:30Erich OceanAnything I can do to speed up Electric re-compilation while I'm editing (dom/props {:class "tailwind-css classes go-here"}) strings, ideally avoiding it altogether?#2023-05-0908:33Erich OceanOkay, I commented out the other examples pages and that dropped it down to about two second to refresh after I hit save. Ideally it'd be less than 100ms, like normal shadow-cljs development.#2023-05-0908:38Geoffrey GaillardTailwind just grep for class names in source files.
Electric is a full compiler.
We are focusing on getting the semantics right. We haven’t optimized compilation speed yet. It’s on the roadmap.#2023-05-0908:43Erich OceanCan dom/props {:class} be set to the value of a e/watch'd atom? Then I could just update that;#2023-05-0908:46Geoffrey GaillardYes this approach would work but, it might not be ergonomic.
Your current approach of commenting out everything you are not working on is ok. This is what I recommend for now.#2023-05-0908:49Erich OceanWill do.#2023-05-0910:39Dustin Getzsorry about the long compile times we don’t have incremental builds yet. it’s a priority#2023-05-0910:41Dustin Getzif you clone the starter app to develop (rather than the examples app with many examples) you will probably be “fast enough” for a few months while we deal with#2023-05-0910:23tobiasHow would I run a js/alert on a button click? I've tried this but it doesn't work:
(e/defn MyButton []
(ui/button (e/fn [] (js/alert "hi"))
(dom/text "Click me")))#2023-05-0910:24tobiasIn the console I see TypeError: Illegal invocation#2023-05-0910:25JHow MyButton is called?#2023-05-0910:30Dustin Getztry window.alert() #2023-05-0910:31Dustin Getz(.alert js/window …)#2023-05-0910:49tobiasThat works! How come that works but js/alert doesn't?#2023-05-0911:15Geoffrey GaillardLooks like the electric compiler fails to resolve js/alert . I’ll log a ticket and we will give it a look. Thank you for the report.#2023-05-0911:11braai engineerDoes Electric handle nested signal functions e.g. tree selectors? I ported my tree picker from Rum which looks roughly like this translated:
(e/defn SimpleTreeNode [{id :xt/id, :account/keys [title children]}]
(e/client
(dom/div
(dom/text title)
(if (seq children)
(e/for-by :xt/id [child children]
(SimpleTreeNode. child))))))
But when I call the root node with: (SimpleTreeNode. {:xt/id :root, :account/name "Root" :account/children […]}) it throws a StackOverflowException at compile time, even when passed an empty list or vector of :account/children.#2023-05-0911:51xificurCIIRC self-refer doesn't work yet. Recursive calls are still active research. Replacing the self call with recur should work for now, but note the behavior is not set in stone yet#2023-05-0911:52xificurCah, it's nested in a for-by, so that won't work either#2023-05-0911:53xificurC(e/def STN)
(e/defn SimpleTreeNode [..] .. (e/for-by [..] (STN. child)))
;; call
(binding [STN SimpleTreeNode] (STN. ...))
#2023-05-0911:53braai engineerI searched for “nested” in channel I see that nested tree signals are not supported, which means I need to render this in a macro. Any gotchas? Tips & tricks?
Yeah if I try to use a normal for it complains that I can’t recur there.#2023-05-0911:55xificurCthis binding hack is what we use for now#2023-05-0912:03braai engineerThanks @U09FL65DK! Is this solvable in future with compiler?#2023-05-0912:05xificurCprobably#2023-05-0914:34Dustin Getz"nested tree signals" what is this#2023-05-0914:36braai engineerMy made up term for recursion of signal fn’s, e.g to render a tree#2023-05-0914:36Dustin Getzoh ok#2023-05-0914:36Dustin Getzit is supported in this manner: https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_recursion.cljc - this is what Peter called the "binding hack"#2023-05-0913:11gravI've adapted the electric-starter-app to PostgreSQL.
The way I've tackled the reactivity is to add an atom with a counter (`(defonce !db-counter (atom 0))`) which gets incremented whenever I mutate the db.
Then I can watch that atom (`(binding [db (e/watch !db-counter)] ...`) and pass it into all functions that need to re-run.
Granted, there's a foot gun in that I need to remember to increment the counter when mutating.
Also, the db argument aren't really used by all the functions that consume it.
Any thoughts on a nicer solution?
https://github.com/grav/electric-starter-app/blob/grav/src/app/todo_list2.cljc#2023-05-0913:12gravBtw, I noticed that the original implementation re-queries the db whenever it wants to render an item - is that necessary?
https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L13#2023-05-0913:16xificurCwe have an https://gist.github.com/dustingetz/1960436eb4044f65ddfcfce3ee0641b7 which uses the same trick you did#2023-05-0913:18xificurCdatomic caches stuff in memory so these queries should be cheap. That's one of the value propositions of datomic#2023-05-0913:28grav> we have an initial guide which uses the same trick you did
Ha, didn't see that one. I guess it was a good exercise to figure that out myself ;-)#2023-05-0914:51Dustin Getzalso see discussion here https://clojurians.slack.com/archives/C7Q9GSHFV/p1682786697009099 and in some other places if you search#2023-05-0914:53Dustin GetzProbably the long term solution is to integrate the router to refresh queries on navigate, and also refresh queries on a configurable timer (say 60 seconds), and force opt-in to get realtime refresh on a custom event for specific realtime regions of the app. The custom event could be on every pg_notify, or hopefully for realtime chat views we are using a document database designed to support realtime collection subscriptions (i.e. not sql)#2023-05-0915:26chromalchemyI sucessfully walked through a http://fly.io deploy of Electric starter app!! 🎉 I’m a devops noob so I was a bit uncertain and trying to being extra carefull, but thankfully it was not too hard! Some notes and deploy questions in thread..#2023-05-0915:32chromalchemyI didn’t have docker installed, and installed docker from Brew (on osx). This tool was limited and insufficient to build container by defualt, and I needed to install Docker desktop.
Maybe other clojure project could work here too.
https://github.com/lispyclouds/contajners
https://github.com/juxt/pack.alpha#2023-05-0915:33chromalchemyWhen I rean the flyctl deploy, it failed the first time. Citing too many restarts. I ran it again and it succeeded.#2023-05-0915:36chromalchemyI was able to use the github secrets config for the CI action.
What is an easy/simple best practice for maintaining a “secret” (api token) from local dev to production?
I am looking into using https://github.com/juxt/aero#hide-passwords-in-local-private-files , but am not sure if that might be overcomplecating things.#2023-05-0916:32chromalchemySince this is kind of generic I posted here also..
https://clojurians.slack.com/archives/C053AK3F9/p1683649479668759#2023-05-0916:33chromalchemyMaybe just duplicate my env variables here is enough..?
https://fly.io/docs/reference/configuration/#the-env-variables-section#2023-05-0916:33chromalchemyvs: For sensitive information, such as credentials or passwords, use the https://fly.io/docs/reference/secrets.#2023-05-0915:39braai engineerHas anybody done any work on macros to port/support Hiccup-based Rum & Reagent components to Electric? It seems that a macro that rewrites [:div.my-class [:h2 cnt “ results”]] to (dom/div (dom/props {:class “my-class”}) (dom/h2 (dom/text cnt “ results))) would go a long way. I know there is Reagent interop.#2023-05-0915:40xificurChttps://github.com/hyperfiddle/electric/issues/32#2023-05-0917:07denik> Since a lot of people have written a considerable amount of code in electric’s current syntax, maybe it would be worthwhile to consider a code-rewriting tool in the future? Could also be a community effort. The highest friction I’ve experienced with electric as an early adopter has been rewriting between electric-domx versions.
https://github.com/hyperfiddle/electric/issues/32#issuecomment-1478393433#2023-05-0919:59chromalchemyIf a want a “ticker” message to update in the dom while a function runs.. eg showing progress of a sequence of api calls.. How would I implement that?
#?(:clj (defonce !query-status) (atom ""))
#?(:clj (defn run-query! []
(reset! !query-status "update message 1)
;; do stuff
(reset! !query-status "update message 2)))
(e/def query-status (e/server (e/watch !query-status))
(e/defn orders-view []
(e/client
(ui/button
(e/fn []
(e/server (run-query!))
(dom/text "Run query"))
(dom/div
(dom/text "Status: "
query-status))))
It seems the default behavior is that the dom node does not update (from query-status : “reactive signal derived from atom”), until the server function run-query! has completely finished#2023-05-0920:18Dustin GetzWhat you wrote seems fine, the dom should update when the atom swaps not be delayed until the end#2023-05-0920:20Dustin Getzoh, the run-query fn is blocking here#2023-05-0920:21Dustin Getzui/button is also not the right abstraction for this, as the ui/button is expecting a single result to become available at which point it will "remove the callback node from the DAG"#2023-05-0920:22Dustin GetzCan you wait a week or two on this? We have a bunch of new UI controls coming that will help us all think more clearly about problems like this#2023-05-0920:23Dustin GetzIf you do make the query async like (e/offload #(run-query!)) it will probably fix your first problem but then i would expect it to hit the second problem and kill the query half-way completed#2023-05-0921:55chromalchemyOk thanks for context. It’s not critical behavior for me atm. Looking forward to the evolutions!!#2023-05-0921:59chromalchemyI was able to work up a functioning dashboard up this afternoon 🎉!! (with operating server code I already had).
Beautiful to see it working! I like that I can update the code from both sides and have view reloaded. Loving the github CI! Now I don’t have to think (worry) about deployments steps so much already.#2023-05-0922:02chromalchemyElectric gave me confidence to go ahead and deploy a whole APP for a limited use case where our SAS api was just not going to give us a simple query filter we needed.#2023-05-0922:05chromalchemy#2023-05-0923:22Dustin Getzlove it#2023-05-1119:47chromalchemyI am getting an error with my longer query function.
It is basically a do-seq that is going through 1400 orders, doing an api request query on each one, then when it matches a predicate, appending the query results to a server side atom (which is synced to cliend with e/def )
> i would expect it to hit the second problem and kill the query half-way completed
Is this what you were referring to?
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] clojure.lang.ExceptionInfo: Websocket pong timeout.
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at hyperfiddle.electric_jetty_adapter$make_heartbeat$cr13526_block_4__13536.invoke(electric_jetty_adapter.clj:22)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at cloroutine.impl$coroutine$fn__10330.invoke(impl.cljc:60)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sequential.step(Sequential.java:86)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sequential$1.invoke(Sequential.java:109)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.core$absolve$fn__10694$fn__10695.invoke(core.cljc:141)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin.terminated(RaceJoin.java:51)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin$1.invoke(RaceJoin.java:71)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin$2.invoke(RaceJoin.java:86)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sleep.trigger(Sleep.java:63)
2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sleep$Scheduler.run(Sleep.java:35)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] ERROR hyperfiddle.electric-jetty-adapter: Websocket error
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] java.lang.NullPointerException: null
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__13579.invoke(electric_jetty_adapter.clj:76)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__13428.invoke(websocket.clj:155)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6.onWebSocketText(Unknown Source)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:296)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:235)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:152)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:148)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:111)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:169)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:90)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at java.base/java.lang.Thread.run(Thread.java:829)
#2023-05-1119:49Dustin GetzThat looks like you blocked the server reactor with a blocking clojure function#2023-05-1119:50Dustin GetzTry (e/offload #(run-query ...)) to move it to a thread pool#2023-05-1119:50Dustin GetzBe aware that e/offload has a subtle issue that causes over-rendering that we haven't fixed - you probably won't notice but I wanted to note it#2023-05-1120:06chromalchemyOk, will try that. Thanks#2023-05-1121:08chromalchemyIt worked! Server fn completed with only adding that e/offload to call. I hear you on the subtleties and works in progress. This was a minimal mvp for me. I still have yet to explore the api in more depth.
But it was great to see I could do this with minimal syntax.
Nice to see table updating too. Thanks so much for the quick advice.#2023-05-1121:18Dustin GetzCan I see your final code?#2023-05-1121:26chromalchemyhere is the repo#2023-05-1121:26chromalchemyhttps://github.com/GetBitOutdoors/gbo-dashboard#2023-05-1121:26chromalchemyhttps://github.com/GetBitOutdoors/gbo-dashboard/tree/main/src/app#2023-05-1121:49chromalchemyIt’s probably bit overcomplicated for what it’s doing. I thought at first I had to declare the atoms in cljc file, so I factored the clj file functions to be pure and take the atoms as args. But would probably keep them global in in a re-write. Or just break the clj code down so it could be called more atomically from electric and only use e/def, and append to it on the client.#2023-05-0922:36Erich OceanAre there docs anywhere that describe the compilation model? For instance, are e/defn's inlined into their caller somehow, when graph coloring is done? Or is it purely lexical? If I use an e/defn from another namespace, will I have any problems related to separate compilation using it? The problem I'm trying to solve here is to have a library of Electric components that can be separately developed (and especially, styled since that workflow is so slow currently) and then use them to build up apps?#2023-05-0922:37Erich OceanThe UIs I'm working on are at least an order of magnitude more complex than the demos.#2023-05-0922:38Erich OceanI'm willing to put up with difficulties while it's in alpha because I really like the overall approach, especially when paired with Datomic on the back end (which is what we use).#2023-05-0923:03Erich OceanFor example, in the Electric repo, when I comment out all of the pages except the one I'm working on in user_main.cljs, the compile time drops a lot. So clearly, ALL pages/namespaces are being re-compiled when one page is updated—even though they haven't changed. Is there something that requires every e/defn function to be re-compiled in order for Electric to "work", or is this just a missing compilation optimization opportunity?#2023-05-0923:04Erich Oceani.e. during compilation, we could (in principle) memoize the result for each e/defn we're compiling, and if it hasn't changed, would the previous compilation result still be valid?#2023-05-0923:43Dustin GetzFunction by function compilation is not yet possible on the current implementation. We are working on it. The current Electric network planner is implemented as a whole-program optimization pass. Some of the network optimizations are done in this compile-time analysis and the optimizations are non-local, i.e. an efficient network plan cannot be inferred without knowledge of the entire program. That's why, today, the whole Electric program (across all namespaces) is compiled all at once for every change. We are already deep into major Electric internals improvements which will unblock many quality of life improvements like incremental builds and function-by-function builds. We are working towards moving certain network optimizations to runtime (i.e. a JIT compiler, like the JVM). That will enable incremental builds. That's great that you want to build complex UIs with Electric. We, too, are building complex things with Electric.#2023-05-0923:52Dustin GetzAlso, it seems like you are still running demos from Electric master? Please do not do that. Master is not stable right now due to landing the aforementioned changes. There are also lots of undocumented changes, including to the demos. Please fork the starter app https://github.com/hyperfiddle/electric-starter-app#2023-05-0923:53Erich OceanI did fork that, but one of the demos didn't work so I switched to master to get it working.#2023-05-1001:45Dustin Getzwhat exactly is broken? we will fix it#2023-05-1005:24Erich Oceanhttps://gist.github.com/erichocean/8c61a27a3ac3f6b39061c7d4054e748c#2023-05-1005:24Erich OceanThere's a thread about it here: https://clojurians.slack.com/archives/C7Q9GSHFV/p1683597379064019#2023-05-0923:53Erich OceanI'll switch back.#2023-05-0923:53Erich OceanBut back to the original question, thanks for the detailed answer.#2023-05-1006:09HendrikI have a question about the startup of an electric program during development.
Where is the compiled electric program injected into the ring middleware stack?
What I found out from the sources:
start fn user(.clj)/main .
main loads shadow and later starts the server.
shadow compiles cljs from entry user(.cljs)/start.
start calls electric-boot.
because electric functions are macros, they are compiled and because clj and shadow use the same Repl the compiled electric program is available in clj.
and from here: how does it get into the ring stack?.
I like electric, but at the moment it feels a little magic. Of course it is not and I would like to understand, what is happening there :)#2023-05-1010:20Dustin Getzsearch for “jetty” we have an example jetty server which hosts the websocket, each websocket gets an instance of the server DAG#2023-05-1011:15HendrikThanks for your answer. ok there is electric-ws-message-handler and electric-ws-message-handler . I looked into the hyperfiddle source code, but I am not able to figure out how the DAG is arriving there. Is there a def dag (atom) or def ^dynamic dag or how is this achieved? Maybe I am just blind and missing some simple statement?#2023-05-1013:47Dustin GetzToday IIRC the clojurescript build is responsible for both server and client
• the client is compiled all the way down: Electric -> DAG IR -> CLJS -> JS
• the server is compiled by the clojurescript build Electric -> DAG IR, and the server DAG value (IR, think bytecode) is embedded into the javascript artifact. The browser client will bootstrap the server by sending up the DAG value to the server in the first frame of the websocket, and then the server interprets the DAG IR at runtime.
(Yes, that means the Electric frontend is compiled all the way to JS but the Electric backend is only partially compiled and then interpreted)
There may be minor errors in this explanation but this is the gist of it.
We have a branch that bakes the server DAG into the java artifact rather than bootstrapping from the client, i forget our exact plans for this but there will be changes this summer#2023-05-1015:26HendrikCool 🙂 Thanks for this explanation. This answers why I could not find anything. I have one more question. If the DAG is provided by the frontend to the server, would n’t this be a security issue or at least an entry point to a DOS attack? I mean, how does the server check, that the DAG is not manipulated?#2023-05-1015:32Dustin GetzAs I said, this summer this is all getting rewritten, the current strategy is convenient for hot code reloading etc. In the meantime we can help you assert the hash in prod if you think this is a credible attack vector and you are using Electric for something serious.#2023-05-1016:03HendrikCurrently I am using it on a small hobby project to explore and learn about electric. I only use libraries in production, which I understand and where I have a sense for the implications of using x or y. I am not at the point yet, where I understand the electric system and its implications well enough to use it for production. However, I see the potential of electric and your answers on this channel already helped me a lot to get a better understanding of electric. So thanks again for answering my questions 🙂#2023-05-1015:16braai engineerhyperfiddle.electric-ui4/date seems to throw when pressing the “Clear” button in browser’s native date picker (using Brave) because it tries to parse NaN:
core.cljs:200 "date input:" #inst "0NaN-NaN-NaNTNaN:NaN:NaN.NaN-00:00"
12:24:34.349 user.cljs:21 Reactor failure: cljs$core$ExceptionInfo {message: 'Remote error - 1011 java.lang.NumberFormatException: For input string: "NaN"',
e.g.:
(ui/date date-query (e/fn [v]
(prn "date input:" v)
(if (string/blank? v)
(reset! !date nil)
(reset! !date v)))
(dom/props {:placeholder "Date"}))
Fell back to dom/input.#2023-05-1015:22Dustin GetzI ticketed it, you can probably easily PR this if urgent#2023-05-1015:22braai engineerthx, not urgent#2023-05-1019:31Erich OceanWhen I run, e.g. XTDB_ENABLE_BYTEUTILS_SHA1=true clj -A:dev -X user/main, there's this printout shadow-cljs - nREPL server started on port 9001. I assume that's the normal JavaScript browser REPL, correct? If so, how are people getting a REPL up for the JVM (i.e. so they can run one of the comment forms)?#2023-05-1019:33Dustin Getzit is a JVM nrepl#2023-05-1019:34Dustin Getzyou can also jack in from your editor in the usual way to get a REPL and run shadow from inside that, like here: https://github.com/hyperfiddle/electric-starter-app/blob/main/src/user.clj#2023-05-1019:35Dustin Getzuse :dev alias#2023-05-1019:42Erich OceanOkay, thanks, got it working. I needed to execute (def db (xt/db !xtdb)) in the user namespace before the comment forms in app.todo-list would work.#2023-05-1019:48Dustin GetzFeel free to PR if you want to improve something#2023-05-1019:50Erich OceanOne thing that's weird is I initially got the todos I had added in the browser client (there were two), but after adding a third, re-running the same query in the REPL returned the previous results (browser shows three).#2023-05-1019:51Erich OceanEven weirder, running queries in user return no items.#2023-05-1019:52Dustin Getzdid you refresh the db val?#2023-05-1019:53Erich OceanLOL, nope. Idiot moment.#2023-05-1101:58kishimaHello! How can I run the starter app on a REPL in order to perform a REPL Driven Development session using Emacs?
https://github.com/hyperfiddle/electric-starter-app
I’m trying to run cider-jack-in and run the user.clj namespace functions but it either doesn’t run or gives me exceptions#2023-05-1102:00Dustin Getzplease post stack traces, did you set :dev alias?#2023-05-1102:01Dustin Getzalternately you can run from the clj command in the readme which will print a nrepl port #2023-05-1102:04kishimaOops I forgot to set the alias, let me try again#2023-05-1102:05kishima#2023-05-1102:05kishimaHmmm, didn’t work, let me try using the clj command#2023-05-1102:12kishimaUsing the clj command it works. I’ll check how can I do it using Emacs directly#2023-05-1103:21kishimaCool, I did it! You only need to use -A:dev on the start and run the user namespace indeed#2023-05-1103:21kishimaThanks!#2023-05-1110:07Dustin Getzthe classpath exception is missing :dev alias, make sure you know how to set the alias in emacs cider jack in by editing the cider command string#2023-05-1112:32kishimaYep, I used C-u M-x to do that, thanks again for the help 🙏 #2023-05-1112:33kishimaYep!#2023-05-1112:33Dustin Getzok great#2023-05-1112:40kishima#2023-05-1108:32Adrian SmithSomething I'm noticing is it so easy to put network IO in-between normally quick local interactions like scrolling, mouse move etc
It is quick but I think to get the smoothness required for it to be not noticeable - I'm going to need to start thinking about patterns for optimistic updates, are there any good examples of this yet?#2023-05-1108:58xificurCCan you give a more concrete example of a problem you see and want to solve?#2023-05-1109:13Adrian SmithHere I have a dom element that takes its position from a server side atom
In the mousemove event I'm updating the server side atom:
https://github.com/slifin/mind-meadow/blob/main/src/mind_meadow/core.cljc#L32
Because it's easy to make this multiplayer is to just do client->server->client on every frame of movement, but at higher movement speed the updates are not keeping up:#2023-05-1109:15Adrian Smithmy assumption is if I update optimistically on the client the dom "slipping" will stop happening, but just need to think carefully about a pattern to update the local state and then update the remote state on mouseup#2023-05-1109:16Adrian Smithor maybe keep updating remote state on mousemove but don't read the dom top/left from it#2023-05-1109:40xificurCthe ui4 controls are solving a similar pattern, e.g. typing into the input you want to sync to the server ASAP but you want to see the typed text before the server roundtrip.
These patterns are being worked on currently and some helpers / changes will land in the next weeks. If you need help more quickly let us know#2023-05-1109:49Adrian SmithI'll check those out, thank you 🙂#2023-05-1109:31braai engineerIs it supposed to be super slow when changing from a complex view to a simple view (using routing like in Datomic browser)? It feels like Electric is “breaking down” a lot of things. Does it have something to do with streaming the DAG over the wire? (I haven’t checked network) Or do I just need to do an advanced production build? (I am not emitting any logs.)
Electric lags a lot when I navigate to a complex view and then immediately try to navigate to a simple view before the view has finished rendering.#2023-05-1109:36xificurCteardown is slow currently, it's on our radar#2023-05-1109:39braai engineerDoes it have something to do with cascading Missionary signal cancellation?
I have a moderately complex app and it is already a performance issue. Are there any tricks to hide the current view, load the new view and delay the teardown or do it in the background? Or a way to mark a subtree as “stale” so Electric stops updating that view.#2023-05-1109:40braai engineer(by moderately complex: it takes ~5s to compile on my late-2019 MBP16)#2023-05-1109:43xificurCWe believe so. Leo is working on changes there that might help or unblock helping this.
No tricks come to mind. Is the performance an issue even on a production build?#2023-05-1110:03Dustin Getzwe’re going to fix it, we haven’t gotten to it yet#2023-05-1110:04Dustin Getzactually it might already be fixed in the next version#2023-05-1110:04Dustin Getzunreleased#2023-05-1110:05braai engineerHaven’t tried on a production build yet - this is all in local dev.
Interestingly, it’s only teardown that’s slow, whereas “setup” is fast.
Looking forward to incremental builds, too 🙂.#2023-05-1111:08Dustin GetzLeo confirms that unmount lag may (may) be solved by the next runtime version which is currently in test#2023-05-1121:02VincentDeveloping in electric is delightful. I needed to figure out a few idioms but now it feels like i'm sailing#2023-05-1201:32markaddlemanIf you start your electric app with a hash at the end of the URL, you’ll end up breaking electric with DOMException: Failed to construct 'WebSocket': The URL contains a fragment identifier (''). Fragment identifiers are not allowed in WebSocket URLs. In my case, the URL is http://localhost:8081/##2023-05-1201:39Dustin Getzthanks will ticket it#2023-05-1520:54Dustin GetzSo this is fixed but it is not a good time to cut a release. We may (may) try to cut a bugfix release this week if it is not too much work#2023-05-1601:17markaddlemanNo worries on my part now that I know to step over the landmine ☺️#2023-05-1209:00BenjaminHow do I eval electric code at the repl?
(def !state (atom []))
(e/def state
(e/server
(let [flow (e/watch !state)]
(m/eduction (map inc) flow))))
How can I for example eval state to see what it is the value?#2023-05-1209:06xificurC(e/run (println state)) would do the trick. Your example doesn't typecheck though, what are you trying to achieve?#2023-05-1209:07BenjaminMy current goal is to have a state atom holding commets for my blog website. I would like to filter on the server depending on the http request url in the end.#2023-05-1209:09xificurCok, and the sample above, where you just trying to increment something? The first issue is you seem to be trying to increment a vector#2023-05-1209:09Benjamintrue#2023-05-1209:10xificurCif the purpose of the exercise was to try incrementing a number you can just call inc on the flow directly, no need for m/eduction#2023-05-1209:11xificurCyou need missionary interop when working with flows. (m/watch !state) returns a flow (note the m/). A flow is a pure value, a recipe for spawning a process. (e/watch !state) is (new (m/watch !state)) where new is how you bridge missionary flows into electric. At that point you can write electric code, e.g. (inc flow)#2023-05-1209:12Dustin Getzhttp://electric.hyperfiddle.net#2023-05-1209:13Benjamin(def !state (atom 0))
(e/def state
(e/server
(inc (e/watch !state))))
(e/run (println state))
where does it print to ?#2023-05-1209:14xificurCe/run runs everything (client and server) locally. So, in the REPL you send it to#2023-05-1216:22chromalchemyI’m getting a :infer-warning warning in the cljs build. Not sure what it means.#2023-05-1216:22chromalchemySomething with cljc file not dispatching the server code correctly?#2023-05-1216:23chromalchemy=> :watching
------ WARNING #1 - :infer-warning ---------------------------------------------
File: /Users/ryan/dev/gbo-dashboard/src/user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Cannot infer target type in expression (. inst__30192__auto__ ound)
--------------------------------------------------------------------------------
9 | (binding [hyperfiddle.electric-dom2/node js/document.body]
10 | (app.dashboard/gator-orders.))))
11 |
12 | (defonce reactor nil)
--------------------------------------------------------------------------------
#2023-05-1216:26chromalchemyI don’t think my user.cljs has changed from the starter, other than re-naming the app.
Here is the cljc file http://user.itis calling.
https://github.com/GetBitOutdoors/gbo-dashboard/blob/main/src/app/dashboard.cljc#2023-05-1217:05Dustin Getzi think this means some JS interop call needs a ^js type hint #2023-05-1217:06Dustin Getzthis can be confounded by Electric not always perfectly propagating type hints and metas (i think these are just bugs we haven’t tracked down yet)#2023-05-1217:07Dustin Getzif you think a type hint should work and it doesn’t, you can extract an ordinary clojure function and type hint the argument in ordinary clojure i.e. (defn wrap-foo [^js x] (.interop-call x))#2023-05-1217:07Dustin Getz@U09FL65DK will correct me if i’ve made an error#2023-05-1217:09Dustin Getz“ound” is an off by one error in a substring on an error message if it helps, i guess we haven’t fixed that yet either #2023-05-1217:17chromalchemy💯
I changed
(dom/div
(ui/long
(clojure.math/round days-back)
(e/fn [v]
(e/server
(reset! !days-back v)))))
to
(dom/div
(ui/long
days-back
(e/fn [v]
(e/server
(reset! !days-back (clojure.math/round v))))))
And I’m not getting warning now!
Looks like (clojure.math/round ...) was in the client code. Giving “ound” in the repl message.#2023-05-1217:18chromalchemyI’m surprised it ran at all considering the nature of that code error.#2023-05-1217:51Dustin Getzi can’t explain that, i will log a ticket#2023-05-1517:54braai engineerNot sure how that ran because clojure.math/round is not defined in the Cljs runtime, only in Clj (unless I am mistaken).#2023-05-1502:42wickedevIn electric app, I've noticed that my device's screen goes white when it goes to sleep and comes back. I was wondering if you could provide any guidance on how to fix this issue#2023-05-1510:30Dustin Getztoday the client restarts when the websocket reconnects. the fix for this is for the electric runtime to resync state on reconnect. this is indeed possible and we did a first pass, which has not landed due to an issue related to how the electric protocol works. the issue will be resolved in an upcoming revision of the runtime (the next electric runtime version is already implemented and in test right now, i don’t recall if the protocol is already improved or if that comes after)#2023-05-1515:07tlonistthat's nice to hear! Do you have any estimation on when the change will be applied?#2023-05-1515:28Dustin Getzthe change needs to be applied carefully, so probably not for a couple months#2023-05-1517:51braai engineerIn this signal fn I am catching the form’s submit event, conj’ing the value of the text input to messages and calling (reset! !text "") to clear the input.
When I run it and type some text, I can click the button to conj a message and clear the input, but when I press Enter (which also submits the form), the message is added but the input is not cleared. Is this because a new text signal value is being produced on keyup?
(e/defn TestApp []
(e/client
(let [!text (atom "")
text (e/watch !text)
handler (e/fn [e]
(e/discard
(.preventDefault e)
(reset! !text "")
(prn "handler" e)
(e/server (swap! !messages conj text))))]
(dom/div
(dom/form
(dom/on "submit" handler)
(ui/input text (e/fn [v] (reset! !text v)))
(dom/button
(dom/props {:type "submit"})
(dom/text "Send")))
(e/for [msg messages]
(dom/div
(dom/text msg)))))))#2023-05-1518:31Dustin Getztry putting a print inside the ui/input callback (in addition to the print in the handler that you already have). If indeed the submit handler runs first and the input handler runs second (for the same interaction) try adding (.stopPropagation e) in the submit handler. (I would not consider this idiomatic but first lets explain what is happening)#2023-05-1518:39braai engineerAdding a print in text input handler shows no event when pressing Enter key.
Adding (.stopPropagation e) to submit handler has not additional affects. Same outcome - weird. I thought there would be a new signal value but equal to prev value.#2023-05-1518:39braai engineer#2023-05-1518:40Dustin Getzput (println text) above the ui/input#2023-05-1518:42braai engineerOK. Sequence of events when typing “abc <enter> <space>”#2023-05-1518:47Dustin Getz"Adding a print in text input handler shows no event when pressing Enter key." this doesn't match the screenshot IMO, i'm not exactly sure where we're not in sync#2023-05-1518:48Dustin Getzpressing enter fires handler first and then input callback second#2023-05-1518:48Dustin Getzoh im misreading it#2023-05-1518:48braai engineerHere is code:
(e/defn Jaymo []
(e/client
(let [!text (atom "")
text (e/watch !text)
handler (e/fn [e]
(e/discard
(do
(.stopPropagation e)
(.preventDefault e)
(reset! !text "")
(prn "handler" e)
(e/server (swap! !messages conj text)))))]
(dom/div
(dom/form
(dom/on "submit" handler)
(println "text:" text)
(ui/input text (e/fn [v]
(prn "input" v)
(prn "val:" (.-value dom/node))
(reset! !text v)))
(dom/button
(dom/props {:type "submit"})
(dom/text "Send")))
(e/for [msg messages]
(dom/div
(dom/text msg)))))))#2023-05-1518:49braai engineerOK so the submit handler fires, adds the message, resets! !text, but then because the ui/input handler also fires, it overrides the newer empty string value?#2023-05-1518:50Dustin Getzthere are too many code snippets now for me to keep track of but yes that was my theory#2023-05-1518:50braai engineer^ that is the latest. No code changes, just more prn’s.#2023-05-1518:50Dustin GetzstopPropagation seems to be working, it prevents the input handler#2023-05-1518:51braai engineerI don’t understand why (reset! !text "") in the submit handler is not clearing the input after submission (with or without stopPropagation).#2023-05-1518:52Dustin Getzoh, ha, i think its because the input is focused#2023-05-1518:53Dustin Getzthe ui/input is specifically coded to prioritize its local state when it is focused#2023-05-1518:53Dustin Getzto prevent a realtime multiplayer update from rewriting the field while you are trying to type
example: https://electric.hyperfiddle.net/user.tutorial-7guis-2-temperature!TemperatureConverter#2023-05-1518:55braai engineerAh, bind-value .#2023-05-1518:57Dustin GetzWe have not considered this pattern where the enter event bubbles out of the input up to the form to submit the form. Crud forms are an active work area for us currently - if you're looking for a quick fix i recommend not listening to the submit event (i.e. not listening to the same event in two places), will the pattern here work for you? https://electric.hyperfiddle.net/user.demo-chat-extended!ChatExtended#2023-05-1519:09Dustin Getz(added link to TemperatureConverter tutorial demoing multiplayer-stabilized ui/inputs)#2023-05-1519:39braai engineerhmm as I see it I’m not listening for the same event in two places: one is to change !text (“input”), and the other is for form submission, which are distinct events IMO and a common way to handle form submissions.
~Even if I use a plain ~~~ (I forgot to set :value text in props.
I find the temperature example surprising because when I focus, local state sticks around but reverts on blur unless I changed it, which overrides the other input (this seems uncommon to me).
For “multiplayer” text inputs I would rather capture the last state in a conditional “editing” input explicitly instead of Electric making assumptions about what is latest – value should be value and/or there should be some explicit localState.
Would prefer if the state-handling was more explicit, e.g. by rendering a fresh stateful input that captures the previous value while editing and either discards or commits my edits.#2023-05-1519:42braai engineerThis seems to work fine:
(dom/input
(dom/props {:type "text" :value text})
(dom/on "change" (e/fn [e]
(let [v (.. e -target -value)]
(prn "input" v)
(prn "val:" (.-value dom/node))
(reset! !text v)))))#2023-05-1519:44Dustin GetzI read it twice, that's a lot of subtle things to unpack#2023-05-1519:44Dustin GetzIs your issue resolved?#2023-05-1519:45braai engineeryes 🙂 works fine if I just use dom/input directly with dom/on “change” or “input” event (“input is more modern)#2023-05-1519:46Dustin Getzok#2023-05-1519:48braai engineerluckily state management is the simplest thing in all of web programming#2023-05-1519:49Dustin Getzi've noted your surprise at the ui/input behavior under concurrent edits#2023-05-1519:50Dustin Getzas well as the gotcha with the focused state taking precedence over the external clear#2023-05-1519:54braai engineerSomething like this:
(e/defn TextInput [value on-change]
(let [!dirty? (atom false)
dirty? (e/watch !dirty?)
!local (atom value)
local (e/watch !local)]
(dom/input
(dom/props {:type "text" :value (if dirty? local value)})
(dom/on "input" (e/fn [e]
(->> (-> e .-target .-value)
(reset! !local)
(on-change))))
(dom/on "focus" (e/fn [e] (reset! !dirty? true)))
(dom/on "blur" (e/fn [e]
(reset! !dirty? false)
(on-change local))))))
(e/defn Jaymo []
(e/client
(let [!text (atom "")
text (e/watch !text)
handler (e/fn [e]
(e/discard
;(.stopPropagation e)
(reset! !text "")
(.preventDefault e)
(prn "handler" (js->clj e))
(e/server (swap! !messages conj text))))]
(dom/div
(dom/form
(dom/text "value:" text)
(dom/on "submit" handler)
(println "text:" text)
(TextInput. text #(reset! !text %))
(dom/button
(dom/props {:type "submit"})
(dom/text "Send")))
(e/for [msg messages]
(dom/div
(dom/text msg)))))))#2023-05-1519:55braai engineerbasically, propagate latest value on every “input”, but use local state while dirty.#2023-05-1519:55braai engineerdoes not suffer from any unexpected Enter / submit event issues. submit means submit. input means input.#2023-05-1519:57braai engineer(not sure if final on-change needed on blur, esp. if it’s not looking in .-value.)#2023-05-1520:00braai engineerHmm, OK I see the inherent focus/blur problem because the form submission is happening before the blur event.#2023-05-1520:17Dustin GetzI mean you can do that if you want, the purpose of the ui namespace is to abstract over this problem (because Hyperfiddle the crud spreadsheet cannot have users writing this code)#2023-05-1520:19Dustin GetzOh sorry you're saying TextInput is an alternate abstraction i guess#2023-05-1520:20Dustin GetzI dont really want to get into it, it would take an hour to unpack whether or not that abstraction is suitable. We spent a lot of time getting ui4 namespace to where it is today#2023-05-1520:20Dustin GetzIt's all getting iterated as we speak anyway#2023-05-1520:23braai engineerNo worries. I see this problem is not unique to Electric and that a common suggested solution is to catch the “submit” event but only actually submit on the next frame via setTimeout(do_submit, 0);
(and/or to force blurs on all input fields, which is kind of gross, but maybe not so crazy.)
I tried to the submit handler to a next frame via (m/sleep 0), but couldn’t get it working in the e/fn. I tried e/task->cp.#2023-05-1520:24Dustin Getzdo you have a link to the problem where that is the suggested solution?#2023-05-1521:24braai engineer• https://stackoverflow.com/a/4482662/198927
• https://stackoverflow.com/a/28124056/198927
When I say “common” I couldn’t tell you how many ppl think this is a good idea, but I have had to do this before when dealing with cascading events.
Catching Enter does seem to be a good way to handle this, and then not track the form submission event. So sorry if this wasted time, but it was useful to me to understand the problem.#2023-05-1522:53Dustin Getzthanks, ok i understand now#2023-05-1523:14VincentHi I hope this isn't a huge curve ball (!) but i was wondering about Sever-side rendering. Mainly so there would be something indexable by Goog at le various endpts#2023-05-1600:34Dustin Getzi believe googlebot runs javascript (it is mechanized chrome), but not sure what the latest seo guidance is#2023-05-1600:35Dustin Getzas to SSR, it’s possible and it will be just as fast as the state of art JS framework SSR which is also based on signals rendering. however this is definitely not a priority as our economics come from enterprise apps#2023-05-1600:36Dustin Getzmaybe we do it next year if it is easy (and it might be really easy, once Electric is mature and running on nodejs etc)#2023-05-1605:17Vincentgroovy. i feel like a single pass of some components would not be complex. more "up to date" pages would require some think.#2023-05-1605:44Adrian Smithhttps://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html an approach from the video game world for managing high frequency input#2023-05-1610:33Dustin Getzthe ui4 namespace implements an optimistic update strategy like this#2023-05-1708:45JHi! I create a https://github.com/jeans11/electric-datomic-starter (based on the electric-xtdb-starter). I notice a little mistake when there are multiple clients. Looks like the db is not the same between the two clients (cf. video in thread)#2023-05-1708:45J#2023-05-1709:19xificurCI suspect each client creates a new tx-queue reader and the first one that manages to .take from the queue wins. You might need to turn the db queue reads into a singleton. Maybe toplevel (e/def db (new (db/latest-db> user/!conn user/!tx-queue)))#2023-05-1709:22JNevertheless, the !tx-queue is like the !conn , both are created once.#2023-05-1709:22JBut I understand your point#2023-05-1710:20JTried (e/def db (new (db/latest-db> user/!conn user/!tx-queue))) on toplevel but same result.#2023-05-1710:58xificurChow do you run datomic? I want to test this locally#2023-05-1711:02JIn memory.#2023-05-1711:04JI have tested with the transactor. Same result#2023-05-1711:05xificurCrunning main I get
Execution error (ExceptionInfo) at datomic.error/raise (error.clj:70).
:db.error/db-not-found Could not find db in catalog
Did you do any setup I have to?#2023-05-1711:44JOups sorry! I create the database manually and transact schema after the connect.#2023-05-1714:14xificurCcan you provide the exact code you use to initialize? To have the same setup and prevent errors#2023-05-1714:25JI updated the repo. Now just launch user/main#2023-05-1718:32jjttjjI just started playing with electric and came here to ask how to setup a datomic connection so that things would be updated reactively when it changes, thanks for this, it's exactly what I was going to ask!#2023-05-1720:07xificurCdidn't get time to look at this, I'll try tomorrow. Thanks for the commit J!#2023-05-1720:10JNo problem#2023-05-1808:03xificurCI sent you a PR with a fix#2023-05-1810:17JThanks @U09FL65DK you rocks!#2023-05-1815:01jjttjjThanks both of you, I've been using this as a guide to use a datomic db to keep state for my electric app, now with the new PR changes.
This isn't related to the problem described (it happened before and after I applied the PR changes) but I'm curious about this.
Assuming db here is bound to an atom containing the latest datomic db value, which should be reset on each tx, is there an obvious reason that d/entity version isn't triggering a ui refresh?
(let [current-chat
;; doesn't work, a change to the user/current-chat doesn't trigger a ui change
(e/server
(-> (d/entity db [:user/id 0])
:user/current-chat
:chat/id))
;; this does work (a change to the current-chat is reflected in the ui)
#_(e/server
(->
(d/pull db [{:user/current-chat [:chat/id]}] [:user/id 0])
:user/current-chat
:chat/id))]
...)
#2023-05-1815:16xificurCit's a duplicate of this https://github.com/tonsky/datascript/issues/433 in datomic. I think @U09K620SG logged a ticket with the datomic team#2023-05-1815:35jjttjjAh, thanks!#2023-05-1816:04Dustin Getzhttps://ask.datomic.com/index.php/859/equality-on-d-entity-ignores-db?show=859#q859#2023-05-1709:04braai engineerHow do I make an Electric component (signal function) that takes an e/fn and runs it correctly? I have a tree picker component that takes a value and an on-change handler, like so: (e/defn TreePicker [value on-change] …).
I want the on-change handler to do do something on the server, but the following does not seem to work:
(dom/div
(TreePicker. value (fn [new-value] (e/server (side-effect! node new-value)))
Do I need to call (new (e/fn [] …) ?
(dom/div
(TreePicker. value (fn [new-value] (new (e/fn [] (e/server (side-effect! node new-value)))))
How do I support signal fn’s as args in my Electric “components”?#2023-05-1709:07xificurC(fn [new-value] (new (e/fn ... this is undefined. It's like writing clojure code in a java method. You're writing electric code in a clojure function. Does that make sense?#2023-05-1709:07xificurCif you need electric code it needs to be an e/fn. You can pass it as an argument and call it with new#2023-05-1709:08xificurC(TreePicker. value (e/fn [new-value] (e/server (side-effect! node new-value))))#2023-05-1709:08braai engineerOK so I can pass an e/fn as my SignalArg and call (new SignalArg arg1 arg2 …)? will try#2023-05-1709:08braai engineerthanks!#2023-05-1709:08xificurCand TreePicker has to call (new on-change ...)#2023-05-1709:09xificurCwe also have a convention to uppercase electric functions, makes it easier to remember when to call new
(e/defn TreePicker [value OnChange] ...)
#2023-05-1712:53braai engineer@U09FL65DK why is (fn [new-value] (new (e/fn [] …)) undefined? Can I do (fn [new-value] (new SomeSignal arg1))? Why is that valid? Is it because the e/fn needs to be defined in root, i.e. can’t be built at “runtime”?#2023-05-1714:12xificurCboth of your examples are invalid. You're essentially writing the equivalent of "clojure-in-java"
public void foo() {
(clojure-code-here)
}
but can't see it because electric is a lisp built on top of clojure#2023-05-1714:13xificurCclojure fn = clojure land, electric not available, you dropped a layer#2023-05-1714:15braai engineerYou said earlier that “TreePicker has to call (new on-change …)” where on-change is an Electric fn, but that can’t happen in a normal Clojure fn? So it has to be Electric turtles all the way down?
And it only works because TreePicker is e/defn in Electric land?
Just to confirm, it’s not possible to run a signal fn (if I’m using the right term, or Electric fn) from a normal Clojure function at all?#2023-05-1714:17xificurCyes
yes
yes 🙂#2023-05-1720:28Dustin Getzcalling into a reactive function from a non-reactive function hits the function coloring problem - like async/await#2023-05-1723:34braai engineerI have a floating picker component that pops up when you click on a particular column for any row in a table. Tricky thing is closing any open pickers when I click on the surrounding div. Annoying to keep track of all !open-pickers (atom #{123 456}) in some global state. Gotta pass the state around, swap! when you click, propagate state back down - kinda messy but at least it’s central.
There are tricks like using the focus/blur events on a text input inside the picker and then preventing propagation of any click events inside the control. So if you click outside, the control blurs and you close the picker. But there are always edge cases and you have to do a cowboy dance to maintain focus in the text input.
Is there a more way convenient way “reach into” Electric components and reset! an atom in all of them, or send them all a signal conveniently? Or maybe “spawn” a new picker when you click on the column that can, I dunno, somehow subscribe to other click or focus events, so it can be closed?
Probably a tale as old as time for floating controls.#2023-05-1801:12Dustin Getzthe question is how to close an open picker by clicking away?#2023-05-1807:17xificurCIIUC the question is how to close all open pickers in 1 swoop, correct? How about
(def !picking? (atom false))
(e/def picking? (e/watch !picking?))
and have the pickers wrapped with a (when picking? ...)? Then you can close all of them with a single (reset! !picking? false)#2023-05-1807:22xificurCI hear your pain with the focus handling, our pickers are also struggling through this. I don't recommend looking at them right now since they are midway through a rewrite and rely on some undefined behavior, but the current strategy there is to have a div covering the entire screen with an increased z-index with the picker on top of that. Then clicks outside the picker land in this div and you can unmount the picker#2023-05-1810:38Dustin Getzyou could also try datascript for top level state rather than lots of atoms#2023-05-2120:36braai engineerUpdate on this: current solution is to catch any clicks on surrounding (background) divs, and closes any open flaoting pickers. Any open floating pickers must call (.stopPropagation e) on click to prevent the background from closing open pickers, but naturally this means you have to keep track of all open pickers globally so the background divs know what to close.
It would be neater if the DOM allowed a way for divs on top (z-index wise) to be notified of clicks outside, or to have something like focus events on divs that wasn’t tied to text inputs, because then the component could close itself when it went “out of scope.”#2023-05-1815:01jjttjjThanks both of you, I've been using this as a guide to use a datomic db to keep state for my electric app, now with the new PR changes.
This isn't related to the problem described (it happened before and after I applied the PR changes) but I'm curious about this.
Assuming db here is bound to an atom containing the latest datomic db value, which should be reset on each tx, is there an obvious reason that d/entity version isn't triggering a ui refresh?
(let [current-chat
;; doesn't work, a change to the user/current-chat doesn't trigger a ui change
(e/server
(-> (d/entity db [:user/id 0])
:user/current-chat
:chat/id))
;; this does work (a change to the current-chat is reflected in the ui)
#_(e/server
(->
(d/pull db [{:user/current-chat [:chat/id]}] [:user/id 0])
:user/current-chat
:chat/id))]
...)
#2023-05-1811:04teodorluHi!
CodeMirror and ProseMirror have some built-in capabilities for collaborative editing. But as far as I'm aware, both are strictly client-side Javascript libraries.
Electric doesn't have the "client side only" limitation.
Has anyone tried using Electric to provide reactive network behavior collaborative editing with CodeMirror or ProseMirror?
https://codemirror.net/ | https://prosemirror.net/#2023-05-1819:17Dustin Getzcollab editing demo: https://prosemirror.net/examples/collab/#edit-Example (two tabs)#2023-05-1819:18Dustin GetzI think a CRDT is the right approach for this, I don't see any obvious benefit in reimplementing the CRDT with Electric#2023-05-1819:20Dustin Getzi.e. https://marijnhaverbeke.nl/blog/collaborative-editing.html - the core patterns here do not come for free with Electric#2023-05-1821:02teodorluRight, the merge logic is not trivial at all.
It probably makes more sense to solve that in Electric userspace. Different Electric might have different persistence layers, and might need different kinds of editing interfaces.#2023-05-1820:41jjttjjHi, still very new to electric but really liking it. I'm wondering how to go about doing the following.
I want to click a button on the client that initiates a "stream" of random numbers, with some time delay between them, and eventually terminating.
below the button i want a dom element to show the random numbers, accumulating them as they arrive.
I have it working with e/watch and e/def and an atom
#?(:clj (def !x (atom [])))
(e/def x (e/server (e/watch !x)))
...
(dom/div
(dom/button
(dom/on "click"
(e/fn [e]
(e/server
(future
(dotimes [_ 10]
(let [n (rand-int 100)]
(Thread/sleep 100)
(println (swap! !x conj n))))))))
(dom/text "button"))
(e/for [n x]
(dom/text (str n " "))))
But in the "real" version of this i want to start many such short lived processes on the server that should stream their updates to the client. I'm wondering what's the Electric way to create an "anonymous" flow on the server (without e/def) and stream the results to the client.
I think I've been slowly making progress towards understanding enough to do this, reading electric source code and refreshing my missionary knowledge, but if anyone has a quick example in the meantime that'd be great#2023-05-1820:44Dustin GetzWe're calling this "progress" (i.e. a button that has a progress indicator while it downloads something)#2023-05-1820:47Dustin GetzThe current ui4/button can kinda do this but there is an impedance mismatch, it wasn't really designed for that. We've learned a LOT since throwing that together and this is part of the current active workstream to rebuild the UI namespace#2023-05-1820:48Dustin GetzHere is a hacked together example of a button with progress under the current ui4/button#2023-05-1820:50Dustin GetzI make no claim that this is idiomatic, it surely will evolve, just giving you something that works today#2023-05-1820:51Dustin GetzIt is almost certainly "bad" - the natural way to model this is to have the button's reactive callback return a sequence of reactive values, but unfortunately the button has some weird semantics on that callback, we're taking a look at it like tomorrow#2023-05-1820:56noonianI ran into a similar case when I want to run an electric function on a callback and the function involves network calls/delays that could cause it to outlive the callback fn's lifecycle. I've been using this pattern of having a atom-wrapped boolean to signal when to do something and changing that in a callback. It doesn't read well but it gets the job done
(e/defn Foo []
(let [!do-thing? (atom nil)
do-thing? (e/watch !do-thing?)]
(when do-thing?
(SomeElectricFn.))
(dom/div
(dom/button
(dom/on "click"
(e/fn [e]
(reset! !do-thing? true)))
(dom/text "do the thing")))))#2023-05-1820:56Dustin Getz> outlive the [ui4/button] callback fn's lifecycle
yeah this is what sucks#2023-05-1820:57Dustin Getzthe ui4/button's "callback" will terminate and unmount after the first non-Pending value is seen, which Leo flagged as a terrible idea many months ago#2023-05-1820:57jjttjjVery interesting, thanks to both of you!#2023-05-1821:00Dustin Getzto be clear, dom/on is the actual thing that sucks#2023-05-1821:01noonianElectric is awesome, and it's still early days! It's a gotcha right now but not too bad to work around and I'm sure the proper api's and idioms will emerge over time#2023-05-1821:05jjttjjYeah 2 days in I've been really blown away by how nice it feels to do client/server stuff. Excited to keep learning it and see it evolve#2023-05-1909:41braai engineerTonsky mentions Electric in latest blog post: https://tonsky.me/blog/humble-signals/#2023-05-1911:15braai engineerWhy can’t I e/watch a DataScript conn? It’s an atom, no?#2023-05-1911:22xificurCyes, a datascript connection can be watched. Can you paste a snippet or have a repo to share?#2023-05-1911:23braai engineerCan it be watched on the client, when conn is not on the server, but in :cljs land?#2023-05-1911:24Dustin Getz(e/client (let [x (e/server (e/watch !d-conn))] ...))#2023-05-1911:25Dustin Getzthe connection is an unserializable reference, the watch must colocate with the conn on the server#2023-05-1911:25braai engineerMy mistake - I had a bad conn value caught in a defonce.#2023-05-1911:26braai engineerI was getting “not Watchable” exception on (e/client (binding [db (e/watch conn)] ..., where conn is on client because conn was a bad value.#2023-05-1911:26Dustin Getzmakes sense#2023-05-1913:26braai engineerWould help if the exception showed the unwatchable value, i.e. “nil”. Would have caught it instantly that way (maybe it does and I missed it)#2023-05-1913:32Dustin Getzok, saved this for later#2023-05-1913:25braai engineerAny idea why I would get these frontend exceptions when I submit a tx on the the server?
(AccountPicker.
(:xt/id account)
(e/fn [new-account]
(log/debug "New value:" id new-account)
(e/server
(e/discard
(log/debug "move entry..." id new-account)
(data/bulk-move-entries! !xtdb [id] new-account)))
;(reset! !editing? false)
(swap! !open-pickers disj id)))#2023-05-1913:32braai engineer#2023-05-1913:33braai engineer(the server tx is successful btw. and everything seems to update)#2023-05-1913:33Dustin Getzi need to see the full context, i.e. everything#2023-05-1913:34Dustin GetzCould your FE and BE be out of sync?#2023-05-1913:36Dustin GetzIt might be (new nil) as well#2023-05-2120:39braai engineerHow would I know if BE & FE are out of sync? Electric would know on WS (re)connect, no? I still get these exceptions sometimes and I doubt it’s because BE & FE are out of sync coz they happen even if I do a force refresh on client. There is probably a bug somewhere, but I can’t tell yet if it’s me or Electric. Will try to reproduce cleanly when I have time.#2023-05-2122:32Dustin Getzi can take a look, can you send me a git clone link? will it be easy to run?#2023-05-2322:06braai engineer@U09K620SG I think I found the cause of this: my index.html in transplanted app had,
<script type="text/javascript" src="/main.js"></script>
`instead of
<script type="text/javascript" src="$:hyperfiddle.client.module/main$"></script>
…which for some weird reason worked in local dev (not sure how).
Haven’t verified the errors are totally resolved, but would explain why FE & BE could be out of sync.#2023-05-1914:37braai engineerShould I be doing,
(e/defn MyThing []
(binding [!xtdb user/!xtdb
db (new (db/latest-db> user/!xtdb))] ...)
at top of every signal fn that needs db access, or just once in a parent component? Is there a cleaner way to pass in the node or db signal as an argument?
Will MyThing be torn down every time db changes if it is passed down as argument?#2023-05-1914:59braai engineerThe db/latest-db> helper seems create a new listener for every signal, so each component would have a listener. How would I share this signal between components? Maybe a !db atom that gets updated and e/watch that?
From Electric XTDB starter app:
(new (db/latest-db> user/!xtdb))
Source:
(defn latest-db>
"return flow of latest XTDB tx, but only works for XTDB in-process mode. see
"
[!xtdb]
(->> (m/observe (fn [!]
(let [listener (xt/listen !xtdb {::xt/event-type ::xt/indexed-tx :with-tx-ops? true} !)]
#(.close listener))))
(m/reductions {} (xt/latest-completed-tx !xtdb)) ; initial value is the latest known tx, possibly nil
(m/relieve {})
(m/latest (fn [{:keys [:xt/tx-time] :as ?tx}]
(if tx-time (xt/db !xtdb {::xt/tx-time tx-time})
(xt/db !xtdb))))))#2023-05-1919:52Dustin GetzI don't fully understand your question/problem#2023-05-1919:52Dustin GetzYou can pass dbvals by lexical or dynamic scope, both are efficient, neither will needlessly reboot your components#2023-05-1919:54Dustin GetzDynamic scope is idiomatic in Electric (specifically moreso than in Clojure, where dynamic scope has too many gotchas)#2023-05-2112:03braai engineerHow do I pass DataScript DB values as signal fn arguments? E.g.
(e/defn MyThing [ds-db]
(e/server
(let [results (ds/q '[:find …] ds-db)]
...)))
It seems to want to transfer the DataScript DB over the wire. Or do I just need to define ds-db as nil in :cljs land?#2023-05-2112:50Dustin GetzYou need to "bias" any functions with server dependencies so that they are called from the server, then the database parameter won't transfer. This is a long standing issue that we can fix later this year#2023-05-2112:52Dustin Getzthe rule today is, Electric functions are "called" from one place, which means all of their parameters must be moved to the place where it is called. So for example in (e/client (MyThing. (e/server db)), db will transfer to the client even if it is not actually used on the client inside MyThing.#2023-05-2112:52Dustin GetzThis is wrong obviously and needs to be fixed.#2023-05-2112:53braai engineerHow do I “bias” the function with server dependencies to prevent the transfer of server parameters? Do I need to use a binding at the top of the function?#2023-05-2112:54Dustin Getzby designing it to be called from the server, it is the call site that matters#2023-05-2112:55braai engineerso e/for-by (if looping over stuff) needs to be in e/server? E.g.
(e/server
(e/for-by :xt/id [row [1 2 3]]
(MyThing. db id))
? Does that still work with e/client stuff in MyThing?#2023-05-2112:55Dustin Getz> Does that still work with e/client stuff in MyThing
yes#2023-05-2113:00braai engineerWhat are the performance implications of running all my “row” loops on the server? Does the client still know the playbook? Any additional overheads?#2023-05-2113:00Dustin Getzthe compiler will make it fast#2023-05-2113:02Dustin Getzi think this tutorial attempts to explain what you need to know - https://electric.hyperfiddle.net/user.demo-system-properties!SystemProperties#2023-05-2113:02Dustin Getz"Reactive for details" and "Reasoning about network transfer"#2023-05-2113:05Dustin Getzalso https://electric.hyperfiddle.net/user.demo-todos-simple!TodoList has close to this pattern (except the database is passed by dynamic scope rather than lexical, which we recommend you do but it's not mandatory)#2023-05-1914:59braai engineerThe db/latest-db> helper seems create a new listener for every signal, so each component would have a listener. How would I share this signal between components? Maybe a !db atom that gets updated and e/watch that?
From Electric XTDB starter app:
(new (db/latest-db> user/!xtdb))
Source:
(defn latest-db>
"return flow of latest XTDB tx, but only works for XTDB in-process mode. see
"
[!xtdb]
(->> (m/observe (fn [!]
(let [listener (xt/listen !xtdb {::xt/event-type ::xt/indexed-tx :with-tx-ops? true} !)]
#(.close listener))))
(m/reductions {} (xt/latest-completed-tx !xtdb)) ; initial value is the latest known tx, possibly nil
(m/relieve {})
(m/latest (fn [{:keys [:xt/tx-time] :as ?tx}]
(if tx-time (xt/db !xtdb {::xt/tx-time tx-time})
(xt/db !xtdb))))))#2023-05-2110:06teawaterwirei built a (silly 🙃 ) Electric app - it runs without erros on my machine, but when i deploy it to Fly it gives an error that i have trouble understanding: https://coffee-sign.fly.dev/ (`Reactor failure`)
the source code is here: https://github.com/teawaterwire/CoffeeSign/blob/main/src/app/coffee_sign.cljc
any idea what it could be? 🙏#2023-05-2113:06Dustin GetzRemote error - 1011 Server process crash#2023-05-2113:06Dustin GetzThere should be something in the server logs#2023-05-2113:20Dustin Getzi have it running locally, going to try to rep it locally (TY for github link)#2023-05-2113:35Dustin GetzI repped it using the prod build locally#2023-05-2113:35Dustin GetzERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure
clojure.lang.ExceptionInfo: Unable to resolve symbol: app.coffee-sign/find-contract#2023-05-2113:35Dustin Getz(This will be in your Fly logs)#2023-05-2113:37Dustin GetzBug is this line: https://github.com/teawaterwire/CoffeeSign/blob/a25d526cb59a9076b8bb97741cad59bb3e295d96/src/prod.clj#L3#2023-05-2113:37Dustin Getzchange to app.coffee-sign#2023-05-2113:39Dustin Getzworks in dev by accident because in dev, shadow and your app are sharing the same JVM so the app's server side got loaded by shadow#2023-05-2114:12teawaterwireah amazing! thanks 🙌
i just pushed let's see 🤞#2023-05-2116:21teawaterwireit worked https://coffee-sign.fly.dev/#2023-05-2118:37teawaterwirei'm surprised to see that the in-memory db is not reset after redeploying. or it's maybe because i only changed css?#2023-05-2118:58Dustin Getzperhaps it was a rolling deploy and you caught an old instance?#2023-05-2216:50Benjamin CHaha @U5UCAE37Z, what a fun idea! I'm a fan 🙂#2023-05-2113:54braai engineerWhat is supposed to happen with db symbol if you do this?
(e/defn App []
(e/server [db (e/watch data/conn)]
(ui/button
(e/fn []
(e/server
(e/discard
(ds/q db ...) ;; 1. will this db reference work?
(ds/transact! data/conn ...)))))) ;; 2. won't this transact cause the e/fn to be cancelled?#2023-05-2114:07Dustin Getzyes#2023-05-2114:08Dustin Getzi mean define "work"#2023-05-2114:08Dustin Getzit will resolve#2023-05-2114:10braai engineer*amended by adding transact.
Won’t the e/fn get cancelled mid-execution because data/conn has changed, triggering the e/watch?#2023-05-2114:11braai engineerI am seeing some weird behaviour that I will have to get back to and try replicate.
I have to deref data/conn in the button’s e/fn otherwise the db query seems to return empty, but I will have to verify what exactly is going on here.#2023-05-2114:45Dustin GetzYou're using datascript? The sample app has a working app#2023-05-2114:46braai engineerI’m using both XTDB and DataScript (for semi-ephemeral things like user selections)#2023-05-2210:06braai engineerHow do I force (history/link ...) to render an absolute path? Because it seems to nest parameterised routes for unrelated route paths. E.g. on ::entries page with URL
(history/link [::tx-detail blah])
;; renders this
Renders this link:
/:app.electric!entries/:app.electric!tx-detail/~uuid,'d8df2c90-0c13-4bb4-a608-19a2e9114d3b'
But it should only render tx-detail, like so:
/:app.electric!tx-detail/~uuid,'d8df2c90-0c13-4bb4-a608-19a2e9114d3b'
I tried changing the router number to be different from the current page’s router number, but no difference. Is this expected? Do I need to add a leading slash somewhere?
The problem is that it breaks Cmd+click (open in new tab), but it works if you just click on same page.
This only became a problem now when I added a routing parameter to the ::entries page.
My top-level Page with calls to history/router looks like this:
(case page
::home (history/router 1 (TxList.))
::entries (history/router 3 (EntryList. x))
::selection (history/router 2 (Selection. :sel1))
::accounts (history/router 1 (Accounts.))
::tx-detail (history/router 2 (TxDetail. x))
::income (history/router 1 (IncomeStatement.))
::help (history/router 1 (HelpView.))
(DefaultView. page))
#2023-05-2211:01Dustin GetzLack of support for cmd-click was an oversight, we will fix it#2023-05-2211:02Dustin GetzWill that resolve your issue? If not can you please state it in 1-2 sentences that stand alone#2023-05-2211:05braai engineerYes, depending on what the solution is?
(history/link [::some-absolute ::routing-path 123] …) should render absolute routing URLs because nested URLs don’t work by default, or support a relative? toggle.#2023-05-2211:23Dustin Getz"nested URLs don’t work by default" I don't understand what you mean by this#2023-05-2211:24Dustin Getzour tutorial app uses this router, is the tutorial app broken (other than cmd-click which is indeed broken in tutorial app)#2023-05-2211:39Geoffrey GaillardI confirm what you are seeing is an issue.
https://github.com/hyperfiddle/electric/commit/cecf4e6a1e66f3b05db8fbb0dcecebfe01622463 fixes it. It is available on master.#2023-05-2211:40Dustin GetzAhhhh#2023-05-2211:42Dustin GetzThat fix commit is included in v2-alpha-263-g89da9d11 — 2023 April 8#2023-05-2211:44braai engineerI am currently on v2-alpha-263-g89da9d11.#2023-05-2211:49Dustin GetzDo you have a clone URL so i can take a look#2023-05-2211:50braai engineernot yet. I still need to deploy it. If anyone has a template lying around to get XTDB deployed with a persistent volume on http://Fly.io (I’m sure it’s just as simple as setting the :db-dir but it’s always a song & dance with env vars), that would accelerate things.#2023-05-2213:20Dustin Getzto be clear i want to clone locally#2023-05-2214:33braai engineerI want to impose a global valid-time for XT queries via an as-of date picker (default to now, but can go backwards/forwards in time).
How do I adapt the latest-db> helper to depend on a global atom value that will either be nil (meaning now / use whatever latest tx-time is) or the datetime specified? I’m not sure how to do this in Missionary land.
Here is the helper from the starter template:
(ns xtdb-contrib
(:require [missionary.core :as m]
[xtdb.api :as xt]))
(defn latest-db>
"return flow of latest XTDB tx, but only works for XTDB in-process mode. see
"
[!xtdb]
(->> (m/observe (fn [!]
(let [listener (xt/listen !xtdb {::xt/event-type ::xt/indexed-tx :with-tx-ops? true} !)]
#(.close listener))))
(m/reductions {} (xt/latest-completed-tx !xtdb)) ; initial value is the latest known tx, possibly nil
(m/relieve {})
(m/latest (fn [{:keys [:xt/tx-time] :as ?tx}]
(if tx-time (xt/db !xtdb {::xt/tx-time tx-time})
(xt/db !xtdb))))))
Can I just change this part to use another tx-time instead of the latest ::xt/tx-time?
(if tx-time (xt/db !xtdb {::xt/tx-time tx-time})
(xt/db !xtdb))#2023-05-2215:09Dustin GetzI would refactor latest-db> into latest-t>, and then from the application I would write something like (xt/db !xtdb {::xt/tx-time (or pinned-t (new (latest-t>)))})#2023-05-2215:10braai engineerIs there a clean way to share db value across components because I don’t think xt/db is free (not sure)?#2023-05-2215:10Dustin Getzbind it in dynamic scope once#2023-05-2215:11braai engineerah, OK. Something I have been doing wrong was to bind db in multiple components coz I had the issue with passing db into components, but realize now I can just bind in the root component. I posted a prior question about this.#2023-05-2214:45braai engineerIn Electric XT starter there is (e/def !xtdb) and (e/def db) which are bound via binding.
How can I move a component that depends on these into its own namespace without duplicating the e/def’s, or is this idiomatic? Alternatively I can pass in the DB stuff then it has to be server-biased.#2023-05-2215:06Dustin GetzUse :require? I dont understand the problem?#2023-05-2215:07braai engineerAh, OK. So have a globals namespace and e/def there? Soz my editor was complaining about missing symbols which confused me. Thought e/def had to be in same file.#2023-05-2215:14Dustin Getzeditor intellisense unfortunately cannot be trusted with Electric right now, do what you would do in Clojure and ask us if it doesn't work#2023-05-2216:04braai engineerIs there a convenience trick to bind globals like db in the REPL for convenient eval, without wrapping every call in (binding [db …] (x/q db …))?#2023-05-2216:40Dustin Getz(def db .)#2023-05-2216:22jjttjjSomewhat similarly I was wondering how to pass server-side stateful components to your electric entry point without depending on global vars (and if it's worth doing). I suppose you just pass the system map around as an argument to your electric functions, and initialize it in a #?(:clj ...) branch in the e/boot function#2023-05-2216:41Dustin GetzCan I see an example and also please define precisely what you mean by "system map" (link me to the DI library you're using for example)#2023-05-2216:50noonianI am doing this with integrant by dynamically binding my server deps in my ws-adapter and again in my electric entrypoint component
(defn inject-deps [ws-handler deps]
(fn [write-msg read-msg]
(binding [my-app.inject/*server-deps* deps]
(ws-handler write-msg read-msg))))
Then wrap the electric ws adapter
(defn wrap-electric-websocket [next-handler deps auth-fn]
(fn [ring-request]
(if (ring/ws-upgrade-request? ring-request)
(let [authenticated-request (auth/basic-authentication-request ring-request auth-fn)
electric-message-handler (-> (partial adapter/electric-ws-message-handler authenticated-request)
(inject-deps deps))]
(ring/ws-upgrade-response (adapter/electric-ws-adapter electric-message-handler)))
(next-handler ring-request))))
Then in your main component
(e/defn Main []
(e/server
(binding [my-app.inject/deps my-app.inject/*server-deps*
my-app.inject/xtdb-node (:xtdb-node my-app.inject/*server-deps*)
my-app.inject/db (new (db/latest-db> (:xtdb-node my-app.inject/*server-deps*)))]
(e/client
;; ...
))))#2023-05-2217:04jjttjj@U09K620SG I didn't have a particular DI lib in mind, and generally am not in a rush to use them. Just working on something similar to this: https://github.com/jeans11/electric-datomic-starter/blob/main/src/app/todo_list.cljc and it's somewhat awkard that the app functions there refer to state in the user namespace. Of course they could be moved to a state/system namespace instead, and I think I'm just going to do that
Mainly I'm just idly wondering out loud, the init process reminds me of the never ending clj DI lib debates where many people prefer no global vars. I don't have a strong opinion on it in general but was a little curious if Electric is taking an opinionated stance on this, to just use a var in a namespace for state, or if it's just that the example apps are small/uncomplicated enough component-state-wise where it doesn't matter.
@U052PH695 nice thanks that's useful to see too#2023-05-2217:21Dustin GetzReactive dynamic bindings in Electric are equal in power to DI frameworks (as far as I am aware). There is a lifecycle, there is state, there is dependency tracking, if something changes the change propagates to the right places, there is graceful shutdown. So, in a pure Electric app there is no need for a DI framework, as Electric itself encompasses that use case
Maybe there are missing parts in this vision, we haven't gone too deep in this#2023-05-2217:23Dustin GetzThe user/!db hack is just a quick hack, yes I'd expect a real system to have some sort of entrypoint namespace to do DI such as inject the js/document.body and the database#2023-05-2217:26Dustin GetzHere is an example of one of our real entrypoints, node bindx is a "de-pyramided" macro over binding#2023-05-2217:36Dustin GetzIt could be cleaner, today each Electric binding list must be entirely on one site (client or server), ideally once we fix that, this is just one big long bindx form (which expands into a big pyramid of nested bindings so they can refer to one another, even across client and server)#2023-05-2217:37jjttjjGotcha thanks, it's helpful to see these approaches!#2023-05-2314:41jjttjjIs there a nice way to reuse an e/fn across other e/fns?
;; save-label is an e/fn that i want to reuse with multiple dom/on event triggers
(let [save-label
(e/fn [e]
(if-let [label (not-empty (.. e -target -textContent))]
(e/server (do-stuff! label))
(println "empty label")))]
(dom/on "blur" (e/fn [e] (save-label e)))
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(.preventDefault e)
(save-label e)
(.. e -target blur)))))#2023-05-2314:42markaddlemanI think this will work for you:
(let [save-label
(e/fn [e]
(if-let [label (not-empty (.. e -target -textContent))]
(e/server (do-stuff! label))
(println "empty label")))]
(dom/on "blur" (e/fn [e] (save-label e)))
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(.preventDefault e)
(new save-label e)
(.. e -target blur)))))
#2023-05-2314:42markaddlemanThe calling convention for electric functions is to use new#2023-05-2314:43jjttjjthat does it, thanks!#2023-05-2314:43markaddlemanI guess it’s more than a convention, it’s the law 🙂#2023-05-2314:49jjttjjquestion that's been on the back of my mind, does new correspond directly to some missionary concept? Does e/fn create a task that new essentially calls m/? on (I think that can't be exactly it but maybe is it something like that)? I'm overdue to do a missionary refresher and I've been curious how exactly it relates to electric. I know it's implemented with it and deeply integrated, but I'm not sure where it is directly used vs wrapped.#2023-05-2314:57markaddlemanAs I recall a comment that @U09K620SG made, it has to do with a limitation of clojure to supply the necessary symbol / fn metadata so that the electric compiler can properly identify electric functions in all cases (related to anonymous functions, maybe?)#2023-05-2314:58markaddlemanRegardless, you’re not newing up something in the javascript/java sense. You’re simply letting the electric compiler know that you’re referencing an electric function rather than a regular clojure fn#2023-05-2315:01jjttjjYup that makes sense#2023-05-2316:31Dustin Getznew (thinking from the missionary layer) corresponds to monadic join on missionary flows, which looks something like (m/cp (m/?< (m/?< <<x)) - given a flow-of-flows, it returns a flow, removing one layer. This is the same concept as async/await on promises#2023-05-2316:33Dustin Getzthe fact that new also boots e/fn is revealing compiler internals, e/fn compiles down literally to a missionary flow. the e/fn parameters are compiled down to flows in dynamic scope that the e/fn flow will access#2023-05-2316:34Dustin GetzTLDR
• new is await for flows - (let [x (new (m/observe ...)) ...)
• new boots Electric e/fns - (let [x (new (e/fn [] ...)) ...)
• it turns out these two operators are the same thing due to compiler internals#2023-05-2316:37Dustin Getzah, we have old docs about this - https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_missionary_interop.clj#2023-05-2315:06braai engineerWhen deploying an Electric app to Fly via GH actions, the build alias deps don’t seem to be cached.
#12 [clojure-deps 6/6] RUN clojure -T:build noop # preload build deps
#12 CACHED
# further down:
#19 [build 9/9] RUN clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'9f539d3'"'
#19 1.611 Cloning:
#19 2.874 Cloning:
#19 4.432 Checking out: at 9bd8b8a3c459966954b21c136e7b1084cb5e0fb0
#19 4.458 Checking out: at ba1a2bf421838802e7bdefc541b41f57582e53b6
#19 5.063 Cloning:
#19 6.216 Checking out: at e3e353262072e95ccac314a9b935b1bc42412a40
#19 6.239 Checking out: at 55fb6f63ea3cc5344e67e87d2322570d4dddd3d5
#19 6.669 Downloading: commons-io/commons-io/2.10.0/commons-io-2.10.0.pom from central
#19 6.814 Downloading: org/codehaus/plexus/plexus-interpolation/1.25/plexus-interpolation-1.25.pom from central
#19 6.818 Downloading: org/apache/maven/maven-builder-support/3.8.2/maven-builder-support-3.8.2.pom from central
Is this the right command to trigger fetching :build alias deps?
RUN clojure -T:build noop
It is the right alias flag.#2023-05-2315:34Geoffrey GaillardThis is the right alias flag. The issue is git deps are prefetched but aren’t carried over to the next build step. So they are fetched a second time. Unlike maven deps (Dockerfile has a COPY instruction for .m2).#2023-05-2315:11braai engineerUpdate: resolved by calling (time-literals.read-write/print-time-literals-clj!) in prod.clj. There is probably a #time/date in a namespace that entrypoint depends on.
I’m heavily using https://github.com/henryw374/time-literals/ in an Electric project. Runs fine locally in dev, but when I deploy to http://Fly.io, it throws a java.lang.ExceptionInInitializerError on startup despite calling (time-literals.read-write/print-time-literals-clj!) at top of my app namespace:
2023-05-23T15:03:40.205 app[1781756a99ed89] jnb [info] at clojure.lang.Util.loadWithClass(Util.java:251)
2023-05-23T15:03:40.205 app[1781756a99ed89] jnb [info] at prod.<clinit>(Unknown Source)
2023-05-23T15:03:40.205 app[1781756a99ed89] jnb [info] Caused by: java.lang.IllegalStateException: Attempting to call unbound fn: #'time-literals.data-readers/date
2023-05-23T15:03:40.205 app[1781756a99ed89] jnb [info] at clojure.lang.Var$Unbound.throwArity(Var.java:45)
2023-05-23T15:03:40.205 app[1781756a99ed89] jnb [info] at clojure.lang.AFn.invoke(AFn.java:32)
#2023-05-2315:26Geoffrey GaillardIs the namespace containing (time-literals.read-write/print-time-literals-clj!) at the top actually loaded/required in your prod setup?#2023-05-2315:27braai engineerThanks @U2DART3HA, I just moved that call to prod.clj (+ require) and now it seems to pass health check on Fly.
Now getting 502 due to Connection state changed (MAX_CONCURRENT_STREAMS == 32)! on Fly, which is not Electric-related (I don’t think).#2023-05-2315:33braai engineerHow much memory does my Electric XTDB app need on http://Fly.io? My deployed app ran out of memory (256MB).#2023-05-2315:36Geoffrey GaillardI don’t know how much your app is going to use. 256MB is low for a JVM app anyway. Bumping it to 1 or 2G is not expensive.#2023-05-2315:40braai engineerOK so 512MB seems to have worked, but now I’m getting Missing client program manifest. Must be missing a file…#2023-05-2316:39Dustin GetzElectric needs 512mb, your database likely needs a lot more#2023-05-2316:46braai engineerrn Electric only works with in-process XT. Why is that? Can it poll XT HTTP server so I can put XT on a bigger machine?#2023-05-2316:46braai engineer(and also to share state)#2023-05-2316:49Dustin GetzThat's a question for the XTDB folks#2023-05-2316:50Dustin GetzElectric can integrate any event stream that XTDB provides#2023-05-2316:52Dustin Getzhttps://clojurians.slack.com/archives/CG3AM2F7V/p1677430221688989#2023-05-2317:21braai engineerHas anyone tried to poll XT’s HTTP tx-log and turn it into a missionary stream?#2023-05-2315:36braai engineerDeploying to http://Fly.io via GH actions frequently blocks because Waiting for remote builder fly-builder-wispy-wildflower-9951...
When I run fly deploy on my machine, it seems to release the builder and unblock the GH build job, at which point I can Ctrl+C my local fly deploy.#2023-05-2315:40Geoffrey GaillardWe have seen similar intermittent issues with fly. It’s not related to electric. Maybe have a look at the --remote-only and --local-only fly deploy flags?#2023-05-2315:56braai engineerDeleted my Fly builder machine and retriggered GH deploy job to see if that fixes it.#2023-05-2316:40Dustin GetzTo echo what Geoffrey said - fly goes down a lot#2023-05-2316:41braai engineer(the above seemed to work)#2023-05-2315:40braai engineerWhat does Missing client program manifest mean in prod?#2023-05-2315:46Geoffrey GaillardIt means the client-side code might not have been compiled, or failed to compile. Look at your fly deploy output for cljs compilation errors. Prefix your fly deploy command with NO_COLOR=1 to get the full logs : NO_COLOR=1 flyctl deploy …
When building the client-side code, the cljs compiler generates a manifest.edn file, sitting next to the compiled .js files. In prod, js artifacts are fingerprinted (`app.<SHA256>.js`) for proper cache invalidation.
The server needs to read the manifest file to serve the js file.#2023-05-2315:51braai engineerAh, thanks. I was missing :prod build in shadow-cljs.edn and didn’t see it because previous failure was cached. Surprised it did not error out, just showed:
#19 24.30 Building client. Version: e7d2c50
#19 33.68 shadow-cljs - server version: 2.20.1 running at
#19 33.68 shadow-cljs - nREPL server started on port 9001
#19 33.68 No configuration for build ":prod" found.
#19 34.69 Bundling sources
#19 34.73 Compiling server. Version: e7d2c50
#2023-05-2316:30Geoffrey GaillardAgree. A cljs build failure should always fail the docker build.#2023-05-2315:59Aldo SolorzanoHello, I'm new to Electric and want to give it a try. The situation I have is the following:
• The back-end is already built and has an endpoint Get /events that returns streams SSE
• I want to make a GET request to /events and constantly react and render something based on what the end-point is returning.
The demos I saw render stuff based on the db changes and from what I understand websockets are happening under-the hood. I don't know if electric would be suitable for my case, any guidance on how to start are very helpful.#2023-05-2316:01Aldo SolorzanoHere is the draft code snippet that serves the streams https://github.com/bob-cd/bob/issues/119#issuecomment-1546904931#2023-05-2316:21jjttjjI've been doing something like this with my first electric project and it definitely works. I don't quite have a super great understanding how it works and have been curious about some things related to it.
I've been using the OpenAI api to get streaming responses. So on my server I make an api request to them, and get a SSE response, which I turn into a lazy sequence from of the streaming results.
I keep the state like this:
#?(:clj (defonce !msg-id->frags (atom {})))
(e/def msg-id->frags (e/server (e/watch !msg-id->frags)))
Each streamed fragment I get is conj'd into the atom like:
(swap! !msg-id->frags update resp-msg-id (fnil conj []) frag)
Then in any electric code I can just refer to msg-id->frags it it will reflect the latest changes.
I have been curious if, under the hood, this entire map is sent across the wire every time a change is made, or if just a diff of the old and new map is sent.
Either way, I could imagine for various reasons wanting to just stream something directly from the server to the client. From browsing the source I'm pretty sure this is possible but too advanced for me at the moment, was hoping to learn more of electric via osmosis before attempting it. I think this stuff might be relevant: https://github.com/hyperfiddle/electric/blob/a65ed416fb0ec8320b43b135bf4c5c1df0a6a607/src/hyperfiddle/electric.cljc#L435-L490
For my use case just watching the atom works well for now.#2023-05-2316:23jjttjjAlso, Dustin's answers to my question from a few days ago might be relevant:
https://clojurians.slack.com/archives/C7Q9GSHFV/p1684442698215489?thread_ts=1684442493.333779&cid=C7Q9GSHFV#2023-05-2316:26Aldo SolorzanoThank you very much, this indeed works as a starting point. I'll give it a try, it is good enough for the moment if all the seq is passed, later on we can see how to optimize it if need it. I'll spend some time playing around with the content yous shared.
The example you mentioned is public code that you could share?#2023-05-2316:28jjttjjI'm working on polishing it off to open source as we speak, possibly today or tomorrow#2023-05-2316:31Aldo Solorzanogood luck there#2023-05-2316:45Dustin Getzthere is no diffing on msg-id->frags, the only Electric primitive that performs diffing is e/for-by#2023-05-2316:46Dustin GetzDon't use e/for-event from master (don't use master at all actually), this is WIP. That is a low level API that is hard to use and we are trying to rework before releasing it.#2023-05-2316:48Dustin GetzAs to how to integrate OpenAI streaming responses, there is certainly a better way to do it than accumulating an atom and watching that, can you share some example responses and how you are trying to render them?#2023-05-2316:49Dustin GetzBut to be clear, the atom is fine if it's working for you#2023-05-2317:11Aldo SolorzanoWhat if we use datascript to persist everytime a event is received? and query for the events to render them in the client? similar to the https://electric.hyperfiddle.net/user.demo-todos-simple!TodoList it uses the e/for-by#2023-05-2317:42Dustin Getzthat strategy consumes unbounded memory (same as using (atom []) as a queue)#2023-05-2317:42Dustin GetzIf it works for you, it's fine#2023-05-2317:45Aldo Solorzanohow would you approach the problem? different solutions work fine and probably for the scale of this problem it's okay.#2023-05-2317:50Dustin GetzI need to see some example responses and how you are trying to render them then i'll tell you 🙂#2023-05-2317:51Aldo Solorzanofair enough, I'll code something and come back 🙂#2023-05-2421:14jjttjj> I'm working on polishing [my openai electric demo app] off to open source as we speak, possibly today or tomorrow
> As to how to integrate OpenAI streaming responses, there is certainly a better way to do it than accumulating an atom and watching that, can you share some example responses and how you are trying to render them?
Got my electric demo out here: https://github.com/jjttjj/chatCLJ Definitely still needs some rounds of refactoring which I'll keep doing over time. In particular this streaming response stuff.
Basically on the server I have a function that makes the streaming api request and returns an eduction of message fragments
(into []
(comp (map #(select-keys % [:choices])) ;; I only use this key
(take 4))
(cljgpt.openai/streaming-request example-request))
[{:choices
[{:delta {:role "assistant"}, :index 0, :finish_reason nil}]}
{:choices [{:delta {:content "Why"}, :index 0, :finish_reason nil}]}
{:choices [{:delta {:content " did"}, :index 0, :finish_reason nil}]}
{:choices [{:delta {:content " the"}, :index 0, :finish_reason nil}]}]
I just conj each of these fragments into a map of msg-id->frag, in an atom that is `e/watched, (while also building up a completed message that will eventually be saved)
https://github.com/jjttjj/chatCLJ/blob/78629d13f97522d3e47f590573b9ff51ada5e778/src/chatclj/app.cljc#L181C8-L191
Then I want to render these fragments so when listing all messages for a chat, if one doesn't yet have a content key I lookup the fragments in msg-id->frags and do a (for-by identity ...) to show each fragment
https://github.com/jjttjj/chatCLJ/blob/78629d13f97522d3e47f590573b9ff51ada5e778/src/chatclj/app.cljc#L267C21-L270#2023-05-2421:36jjttjjSo at a super high level in bad "pseudocode", what I'm doing looks like this:
(def db (atom {1 "msg1" 2 "msg2" 3 :placeholder})) ;;<- this is the db in the real app
(let [;;frags isn't in the db, it's only temporary and could have it's scope limited to just the code below
frags (atom {3 "m"}) ;; eventually "msg3"
user-question "what's 1+1"]
(for [[id msg] db]
(if (= msg :placeholder)
(show-fragments id)
(show-msg msg)))
(input
(on :enter-keypress
(fn [e]
(let [msg-id 3
answer-reducible (streaming-request user-question)
;;ie ["a" "b" "c" ...]
]
;; the most straightforward thing
(run! #(swap! frags update msg-id str %) answer-reducible))))))
I guess te question is, is there a way in electric to directly stream something on the server to something atom-like that can be watched on the client?#2023-05-2421:39jjttjjActually, sorry. I realized this was already answered in my earlier slack question that I even linked to above, and I just need to take a serious stab at those methods (I got this working and moved on to other stuff before trying them deeply). But I guess I'll leave this here because it was sort of useful to re-state the situation#2023-05-2716:54Dustin GetzI tried the chatclj app on my machine, I get 404 errors in the log when I submit a prompt. The api key is set as you see in bearers. Have I done something wrong?
clojure.lang.ExceptionInfo: Exceptional status code: 404
body: #object[java.util.zip.GZIPInputStream 0x410d7a00 "java.util.zip.GZIPInputStream@410d7a00"]
headers: {"content-encoding" "gzip",
"server" "cloudflare",
"content-type" "application/json; charset=utf-8",
"alt-svc" "h3=\":443\"; ma=86400",
"strict-transport-security" "max-age=15724800; includeSubDomains",
"cf-cache-status" "DYNAMIC",
"cf-ray" "7cdfc40dd9ca8ccc-EWR",
":status" "404",
"date" "Sat, 27 May 2023 16:52:00 GMT",
"vary" "Origin",
...}
request: {:headers
{:accept "*/*",
:accept-encoding ["gzip" "deflate"],
:user-agent "babashka.http-client/0.1.4",
"Content-Type" "application/json",
"Authorization"
"Bearer sk-qfi...H1tX"}, // Open API key
:body
"{\"model\":\"gpt-4\",\"messages\":[{\"role\":\"system\",\"content\":\"You are a helpful assistant.\"},{\"role\":\"user\",\"content\":\"what is \\\"ring\\\" in category theory?\"},{\"role\":\"user\",\"content\":\"translate \\\"hello\\\" to french\"}],\"stream\":true}",
:as :stream,
:uri "",
:method :post}
status: 404
version: :http2#2023-05-2717:03Dustin GetzOk, I read the thread and looked at the code in the links.
The stated question: "is there a way in electric to directly stream something on the server to something atom-like that can be watched on the client?"
I don't understand the question and I also don't understand why you would ask that, I think there is missing context.
Can we start with the problem – what problem are you facing that makes you ask this question?
Please answer in the language of the end user, please do not answer in the language of programming#2023-05-2914:07jjttjj> I tried the chatclj app on my machine, I get 404 errors in the log when I submit a prompt.
Hmm, just to rule some things out, you have a funded openAI account right? It looks like that request you made is using gpt-4 which I think still requires getting on a waitlist and isn't generally available. I should put this in my readme. When you start a chat with gpt 3.5 do you still get the 404? You should be able to try out api requests here to confirm that they work for your openai account: https://platform.openai.com/playground?mode=chat
> The stated question: "is there a way in electric to directly stream something on the server to something atom-like that can be watched on the client?"
> Can we start with the problem – what problem are you facing that makes you ask this question?
Good point, it's definitely not a user-facing problem. Partly just asking scattershot questions to get up to speed on Electric, and curious if your earlier point of:
> As to how to integrate OpenAI streaming responses, there is certainly a better way to do it than accumulating an atom and watching that, can you share some example responses and how you are trying to render them?
still applied.
I was also a bit "worried" that the entire accumulated sequence of fragments was being sent over the wire each time an new fragment was received on the server, but I think that is avoided due to using e/for-by to iterate over the fragments here: https://github.com/jjttjj/chatCLJ/blob/c875af3fa030459edc60ac8c35681b37eb32897c/src/chatclj/app.cljc#L313-L316
This would be a premature-optimization at this point in any case, what I have works perfectly here. I have done things in the past that required direct server to client ui streaming of large amounts of data, so just a little curious.
I do think I'm at a point now where I have enough context with Electric and a backlog of examples to try out that I've found in the repo that I could make an attempt at this when the need arises and have more specific questions then 🙂#2023-05-2915:31Dustin Getzyes I pay for ChatGPT and have GPT4 access#2023-05-2915:32Dustin GetzAh, I guess paying for the OpenAI API is different than paying for ChatGPT ?#2023-05-2915:33Dustin GetzI now see that it is, and my "free usage has expired" thanks#2023-05-2915:36Dustin Getzyou'll want to wrap that e/for-by in an e/server for (get msg-id->frags id) to run on the server and therefore individual frag to stream; currently the e/for-by is on the client so (get msg-id->frags id) is running on the client, which means (e/def msg-id->frags (e/server (e/watch !msg-id->frags))) the entire atom value is streaming to the client so the get can run on the client#2023-05-2915:37Dustin GetzThe point is that Electric does give you what you need to reason directly about efficient network transfers#2023-05-2915:43Dustin GetzIf your question is: "How do I stream a sequence of server events/delta/frags to the client efficiently":
Two things to consider:
• minimizing network traffic
• cost of diffing
#2023-05-2915:43Dustin GetzMinimizing network traffic can be done with e/for-by as I described a moment ago#2023-05-2915:43Dustin GetzCost of diffing is: if OpenAI is giving us a stream of events/deltas/frags, why would we collect them into an atom with (swap! !msg-id->frags update resp-msg-id (fnil conj []) frag) (specifically conj) only to pass the collection to e/for-by which is going to separate them back into deltas by computing diffs?#2023-05-2915:44Dustin GetzAlso, storing the whole collection in an atom on the server costs unbounded memory, and if that collection gets huge, and you have to diff it back into frags each time it changes, ...#2023-05-2915:45jjttjjAh, the e/server wrapping now makes perfect sense. This worked on the first go and reads very nicely in that it's clear that it only streams the changes#2023-05-2915:47Dustin GetzThe solution here (today) is tricky, it would involve dropping down to missionary and bypassing e/for-by entirely. We're working on an alternate primitive to e/for-by, perhaps called e/for-integrate-by or something which accepts a discrete event stream as an input (thereby skipping the diffing step)#2023-05-2915:47Dustin GetzSo for now I would continue doing what you're doing, as obviously it works well enough despite the theoretical flaws#2023-05-2316:00braai engineerMaybe this is shadow-cljs specific, but GH deployment to Fly does not stop if Cljs build fails due to a missing package.json dependency. This causes CI to deploy a broken image that passes health check.#2023-05-2316:15braai engineerWhere should I add a call to yarn or npm install in the client build pipeline? Seems like my cljs dependencies in package.json are not being installed.
According to Shadow-cljs, docs supposed to call it: https://shadow-cljs.github.io/docs/UsersGuide.html#npm-install#2023-05-2316:31braai engineerI see there is a node-deps step on example app now: https://github.com/hyperfiddle/electric-examples-app/blob/60e2e6f8c34c6397da91af7d067bd977a75240c1/Dockerfile#L1-L4
What’s the latest? Is there a reference Dockerfile for Electric XT?#2023-05-2316:31Geoffrey GaillardAlso look at line 8.#2023-05-2316:32Geoffrey GaillardYour link points to the latest.#2023-05-2316:33braai engineerThis is what I have:
FROM node:14.7-stretch AS node-deps
WORKDIR /app
COPY package.json package.json
RUN npm install
FROM clojure:openjdk-11-tools-deps AS clojure-deps
WORKDIR /app
COPY deps.edn deps.edn
COPY src-build src-build
RUN clojure -A:dev -M -e :ok # preload deps
RUN clojure -T:build noop # preload build deps
FROM clojure:openjdk-11-tools-deps AS build
WORKDIR /app
COPY --from=clojure-deps /root/.m2 /root/.m2
COPY --from=node-deps /app/node_modules /app/node_modules
COPY shadow-cljs.edn shadow-cljs.edn
COPY deps.edn deps.edn
COPY src src
COPY src-build src-build
COPY resources resources
ARG REBUILD=unknown
ARG VERSION
RUN clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'$VERSION'"'
FROM amazoncorretto:11 AS app
WORKDIR /app
COPY --from=build /app/app.jar app.jar
EXPOSE 8080
ARG VERSION
ENV VERSION=$VERSION
CMD java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$VERSION -jar app.jar
Can I replace it with the reference? (for Electric XT)
Note this is for XT, not Datomic.#2023-05-2316:37Geoffrey GaillardYou cannot just replace it, lines 10 to 28 are not relevant outside of the electric-examples-app repo.#2023-05-2316:38braai engineer> [build 7/13] COPY .m2 /root/.m2:
------
Error: failed to fetch an image or build from source: error building: failed to compute cache key: "/.m2" not found: not found
^ also#2023-05-2316:41braai engineercurrently blocked on this. should it be COPY /root/.m2 /root/.m2?#2023-05-2316:43Geoffrey GaillardThere is a .m2 in electric-examples-app, but probably not in your project.#2023-05-2316:43braai engineershould I create it? isn’t that supposed to be my machine’s m2 cache?#2023-05-2316:44Geoffrey GaillardUnless you need to set maven repository credentials, then you don’t need it and can drop the line in the Dockerfile.#2023-05-2316:44braai engineerIsn’t .m2 related to caching maven deps? I’m trying with this step added back:
FROM clojure:openjdk-11-tools-deps AS clojure-deps
WORKDIR /app
COPY deps.edn deps.edn
COPY src-build src-build
RUN clojure -A:dev -M -e :ok # preload deps
RUN clojure -T:build noop # preload build deps
FROM clojure:openjdk-11-tools-deps AS build
WORKDIR /app
COPY --from=clojure-deps /root/.m2 /root/.m2
...
#2023-05-2316:54braai engineerfuuu image size is now 766MB, but it fixes my missing node deps.
now stuck on required namespace "user" is not available (src/user.cljs is committed):
#22 19.42 [:prod] Compiling ...
#22 19.95 -> build target: :browser stage: :configure
#22 19.96 <- build target: :browser stage: :configure (3 ms)
#22 19.96 -> Resolving Module: :main
#22 19.96 The required namespace "user" is not available.
#22 19.96
#22 DONE 20.5s
#23 exporting to image
Build still continues and deploys with missing manifest error.#2023-05-2317:26Geoffrey GaillardMaybe a classpath issue. Maybe some of your prod code references user/something .#2023-05-2317:27braai engineerhmm nope. Here is src/user.cljs:
(ns ^:dev/always user ; Electric currently needs to rebuild everything when any file changes. Will fix
(:require
app.electric ;; my app entrypoint
hyperfiddle.electric
hyperfiddle.electric-dom2))
(prn "loading user namespace")
(def electric-main
(hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
(binding [hyperfiddle.electric-dom2/node js/document.body]
(app.electric/MyApp.))))
(defonce reactor nil)
(defn ^:dev/after-load ^:export start! []
(prn "start")
(assert (nil? reactor) "reactor already running")
(set! reactor (electric-main
#(js/console.log "Reactor success:" %)
#(js/console.error "Reactor failure:" %))))
(defn ^:dev/before-load stop! []
(prn "stop")
(when reactor (reactor)) ; teardown
(set! reactor nil))#2023-05-2317:28braai engineerOh you said prod code…Hmm here is src/prod.clj:
(ns prod
(:gen-class)
(:require
time-literals.read-write
app.electric
clojure.string
;electric-server-java11-jetty10
electric-server-java8-jetty9))
(time-literals.read-write/print-time-literals-clj!) ;; here because complains on startup if in Electric.
(def electric-server-config
{:host "0.0.0.0", :port 8080, :resources-path "public"})
(defn -main [& args] ; run with `clj -M -m prod`
(when (clojure.string/blank? (System/getProperty "HYPERFIDDLE_ELECTRIC_SERVER_VERSION"))
(throw (ex-info "HYPERFIDDLE_ELECTRIC_SERVER_VERSION jvm property must be set in prod" {})))
(electric-server-java8-jetty9/start-server! electric-server-config))
; On CLJS side we reuse src/user.cljs for prod entrypoint
the only thing that changed recently is time-literals.#2023-05-2319:03Dustin Getz> the only thing that changed recently is time-literals.
are you saying this used to work and then stopped working at a specific commit?#2023-05-2316:22braai engineerWould it be possible to run the Electric cljs build and the clj uberjar build in parallel?#2023-05-2316:34Geoffrey GaillardYes#2023-05-2317:10braai engineerWhy would prod shadow-cljs build complain that The required namespace "user" is not available when src/user.cljs is committed?
My shadow-cljs.edn matches https://github.com/hyperfiddle/electric-examples-app/blob/60e2e6f8c34c6397da91af7d067bd977a75240c1/shadow-cljs.edn.#2023-05-2318:04braai engineerWhen I run clojure -X:build build-client locally, prod build seems to hang:
Building client. Version: 0e62a92-dirty
shadow-cljs - server version: 2.20.1 running at
shadow-cljs - nREPL server started on port 9001
[:prod] Compiling ...
TRACE hyperfiddle.electric.impl.env: initial load opentax.electric
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external java.math.RoundingMode
TRACE hyperfiddle.electric.impl.env: loading external java.math.RoundingMode
TRACE hyperfiddle.electric.impl.env: loading external java.math.RoundingMode
# it just waits here forever.
Gonna try with verbose & debug.#2023-05-2318:21braai engineerSo everything runs fine locally, but on GH actions it fails:
#22 [build 10/10] RUN clojure -T:build build-client :verbose true :version '"'fdf113d'"'
#22 1.236 Cloning:
#22 1.667 Cloning:
#22 2.061 Checking out: at ba1a2bf421838802e7bdefc541b41f57582e53b6
#22 2.081 Checking out: at 9bd8b8a3c459966954b21c136e7b1084cb5e0fb0
#22 2.248 Cloning:
#22 2.565 Checking out: at e3e353262072e95ccac314a9b935b1bc42412a40
#22 2.582 Checking out: at 55fb6f63ea3cc5344e67e87d2322570d4dddd3d5
#22 12.79 fatal: not a git repository (or any of the parent directories): .git
#22 14.02 Building client. Version: fdf113d
#22 19.44 shadow-cljs - server version: 2.20.1 running at
#22 19.44 shadow-cljs - nREPL server started on port 9001
#22 19.45 [:prod] Compiling ...
#22 19.97 -> build target: :browser stage: :configure
#22 19.97 <- build target: :browser stage: :configure (4 ms)
#22 19.97 -> Resolving Module: :main
#22 19.98 The required namespace "user" is not available.
#22 19.98
#22 DONE 20.5s
#2023-05-2322:28braai engineerResolved by using the Dockerfile from electric-starter-app, not the commands from electric-examples-app.
So instead of this from electric-examples-app:
RUN clojure -T:build build-client :verbose true :version '"'$VERSION'"'
Use:
RUN clojure -X:build uberjar :jar-name "app.jar" :verbose true :version '"'$VERSION'"'#2023-05-2317:59braai engineerHow long does advanced cljs build typically take? dev build takes ~30s on my machine, but prod seems to take forever (or hangs).#2023-05-2318:06braai engineerTrying now with clojure -X:build build-client verbose=true:
[:prod] Compiling ...
-> build target: :browser stage: :configure
<- build target: :browser stage: :configure (3 ms)
-> Resolving Module: :main
<- Resolving Module: :main (1145 ms)
...
<- Cache write: com/rpl/specter.cljc (129 ms)
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external clojure.lang.PersistentArrayMap
TRACE hyperfiddle.electric.impl.env: loading external java.math.RoundingMode
<- Compile CLJS: user.cljs (17209 ms)
-> build target: :browser stage: :compile-finish
<- build target: :browser stage: :compile-finish (1 ms)
-> build target: :browser stage: :optimize-prepare
<- build target: :browser stage: :optimize-prepare (0 ms)
-> Closure - Optimizing ...
Optimizing CLJS Constants took 1071ms
... ;; hangs here.
#2023-05-2318:08braai engineer…
<- Closure - Optimizing ... (186481 ms)
-> build target: :browser stage: :optimize-finish
<- build target: :browser stage: :optimize-finish (0 ms)
-> build target: :browser stage: :flush
-> Flushing optimized modules
Flushing: main.D52B65600F20EE2C84CB5486269BBD07.js (22972665 bytes)
<- Flushing optimized modules (1723 ms)
<- build target: :browser stage: :flush (2384 ms)
[:prod] Build completed. (223 files, 91 compiled, 0 warnings, 232.74s)
phew OK, advanced build takes about 4 minutes (with debug setting).#2023-05-2318:09braai engineerany tips to speed this up? maybe ditch Specter in client?#2023-05-2318:10braai engineerAside, I get these Missionary warnings on advanced build:
Optimizing CLJS Constants took 1071ms
------ WARNING #1 - -----------------------------------------------------------
File: ~/.m2/repository/org/clojure/clojurescript/1.11.60/clojurescript-1.11.60.jar!/cljs/pprint.cljs:260
--------------------------------------------------------------------------------
257 |
258 | (deftype end-block-t :logical-block :start-pos :end-pos)
259 |
260 | (deftype indent-t :logical-block :relative-to :offset :start-pos :end-pos)
--------------------------------------------------------------------------------
variable G__35634__$1 is undeclared
--------------------------------------------------------------------------------
261 |
262 | (def ^:private pp-newline (fn [] "\n"))
263 |
264 | (declare emit-nl)
--------------------------------------------------------------------------------
------ WARNING #2 - -----------------------------------------------------------
File: ~/.m2/repository/missionary/missionary/b.27-SNAPSHOT/missionary-b.27-SNAPSHOT.jar!/missionary/impl.cljs:161:15
--------------------------------------------------------------------------------
158 | (when (== n (alength (.-result j)))
159 | (let [w (.-race j)]
160 | (if (neg? w)
161 | (try ((.-joincb j) (.apply (.-combinator j) nil (.-result j)))
---------------------^----------------------------------------------------------
variable $fexpr__35624 is undeclared
--------------------------------------------------------------------------------
162 | (catch :default e ((.-racecb j) e)))
163 | ((.-racecb j) (aget (.-result j) w)))))))
164 |
165 | (defn race-join [r c ts s f]
--------------------------------------------------------------------------------
nil
<- Closure - Optimizing ... (186481 ms)
#2023-05-2318:24braai engineerIn Fly build step on GH I get a “fatal: not a git repository” error. Could that be causing my “required namespace “user” is not available issue?
#22 [build 10/10] RUN clojure -T:build build-client :verbose true :version '"'fdf113d'"'
#22 1.236 Cloning: ;; ...
;; ...
#22 2.582 Checking out: at 55fb6f63ea3cc5344e67e87d2322570d4dddd3d5
#22 12.79 fatal: not a git repository (or any of the parent directories): .git
#22 14.02 Building client. Version: fdf113d
#2023-05-2318:41Geoffrey GaillardThis message seems like a false positive. We have seen it and had no impact. I don’t think it is related to your issue.#2023-05-2322:25braai engineerHow do I debug Reactor failure: Remote error - 1011?
Finally got my app deployed to Fly, but when I open the app in browser, I see a brief flash of text and then: Remote error - 1011 org.eclipse.jetty.websocket.core.exception.WebSocketException, which I think means error on server.
The app has not run out of memory on Fly (268MB/496MB) and is still running, but I don’t see any logs on Fly’s monitoring.
First thought is to do a simple (not advanced) client build, because JS stacktrace is inscrutable.
Logback.xml has these two <logger> entries:
<logger name="hyperfiddle" level="DEBUG" additivity="false"><appender-ref ref="STDOUT" /></logger>
<logger name="hyperfiddle.electric-jetty-adapter" level="WARN" additivity="false"><appender-ref ref="STDOUT" /></logger>
#2023-05-2322:43Dustin Getzthe first thing i do is run the prod build locally to repro outside of fly#2023-05-2322:44Dustin Getzi don’t know why it isn’t logging the exception, i don’t think we have seen that issue yet#2023-05-2322:46Dustin Getzwebsocket exception may be different than application exception on server as well, on mobile so can’t look at code path easily for that error#2023-05-2323:07braai engineerRan the prod build locally. It starts, throws a different TypeError, but reactor does not crash:
in.3BE3A164204AC9A7A9881BC3DE98B078.js:1699 Uncaught TypeError: Cannot set properties of null (setting 'X')
at B_ (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1699:290)
at p_ (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1705:57)
at o_.o (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1695:291)
at f0 (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1717:77)
at e0 (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1718:31)
at S_ (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1716:390)
at R_.o (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1713:780)
at f0 (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1717:77)
at e0 (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1718:31)
at S_ (main.3BE3A164204AC9A7A9881BC3DE98B078.js:1716:390)
B
This happens even for a simple view that does nothing other than render a static text message (no DB queries).#2023-05-2323:10Dustin Getzand the dev build works?#2023-05-2323:22braai engineerBoth dev and prod work fine locally. ~They both throw~ Only prod throws the TypeError, but it does not seem to cause any problems. I nee to populate the DB in this folder to be 100% sure tho.
I also did prod with :optimizations :simple - same outcome. Nothing in server logs.#2023-05-2323:23braai engineerhere is the error in ~dev~ prod with optimizations :simple
in.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:6943 Uncaught TypeError: Cannot set properties of null (setting 'prev')
at missionary.impl.Reactor.propagate (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:6943:259)
at missionary.impl.Reactor.event (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:6954:168)
at missionary.impl.Reactor.Process.cljs$core$IFn$_invoke$arity$0 (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:6934:245)
at missionary.impl.Ambiguous.cancel (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7009:157)
at missionary.impl.Ambiguous.walk (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7011:85)
at missionary.impl.Ambiguous.kill (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7008:459)
at missionary.impl.Ambiguous.Process.cljs$core$IFn$_invoke$arity$0 (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:6989:374)
at missionary.impl.Ambiguous.cancel (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7009:157)
at missionary.impl.Ambiguous.walk (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7011:85)
at missionary.impl.Ambiguous.kill (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7008:459)
m#2023-05-2323:30braai engineer@U09K620SG so an interesting thing happens if I start up dev, load up the client (all works as expected in dev - no errors), then I stop dev and start prod while keeping client open (built from same source - should be same hash right? not sure). client then attempts reconnection and sees a reactor failure that might be the same error I’m seeing on Fly, but I can’t be 100% sure and maybe it’s something totally different:
electric_client.cljs:31 WebSocket connection to '' failed:
eval @ electric_client.cljs:31
eval @ Sequential.cljs:29
missionary$impl$Sequential$suspend @ Sequential.cljs:29
eval @ Sequential.cljs:13
missionary$core$park @ core.cljc:162
hyperfiddle$electric_client$connector_$_cr62862_block_0 @ electric_client.cljs:78
eval @ impl.cljc:60
G__59769__0 @ impl.cljc:60
eval @ Sequential.cljs:40
missionary$impl$Sequential$step @ Sequential.cljs:40
missionary$impl$Sequential$run @ Sequential.cljs:59
missionary$core$sp_run @ core.cljc:175
G__59769__3 @ impl.cljc:65
G__46710__2 @ core.cljs:4358
eval @ Sequential.cljs:29
missionary$impl$Sequential$suspend @ Sequential.cljs:29
eval @ Sequential.cljs:13
missionary$core$park @ core.cljc:162
hyperfiddle$electric_client$boot_with_retry_$_cr63073_block_2 @ electric_client.cljs:103
eval @ impl.cljc:60
G__59769__0 @ impl.cljc:60
eval @ Sequential.cljs:40
missionary$impl$Sequential$step @ Sequential.cljs:40
eval @ Sequential.cljs:53
eval @ impl.cljs:205
01:26:08.032 electric_client.cljs:126 Failed to connect.
01:26:08.033 electric_client.cljs:127 Next attempt in 5.5 seconds.
01:26:08.758 websocket.cljs:12 WebSocket connection to '' failed:
shadow$cljs$devtools$client$websocket$start @ websocket.cljs:12
eval @ shared.cljs:324
eval @ shared.cljs:345
01:26:08.758 shared.cljs:305 shadow-cljs - remote-error Event {isTrusted: true, type: 'error', target: WebSocket, currentTarget: WebSocket, eventPhase: 2, …}
eval @ shared.cljs:305
shadow$cljs$devtools$client$shared$remote_error @ shared.cljs:18
eval @ websocket.cljs:29
error (async)
shadow$cljs$devtools$client$websocket$start @ websocket.cljs:26
eval @ shared.cljs:324
eval @ shared.cljs:345
setTimeout (async)
eval @ shared.cljs:341
eval @ shared.cljs:297
shadow$cljs$devtools$client$shared$remote_close @ shared.cljs:17
eval @ websocket.cljs:24
01:26:13.542 electric_client.cljs:107 Connecting...
01:26:13.762 electric_client.cljs:112 Connected.
01:26:13.954 electric_client.cljs:66 WebSocket is already in CLOSING or CLOSED state.
hyperfiddle$electric_client$send_BANG_ @ electric_client.cljs:66
eval @ electric_client.cljs:69
hyperfiddle$electric_client$send_all_$_cr62775_block_1 @ electric_client.cljs:69
eval @ impl.cljc:60
G__59769__0 @ impl.cljc:60
eval @ Ambiguous.cljs:262
missionary$impl$Ambiguous$ready @ Ambiguous.cljs:264
missionary$impl$Ambiguous$boot @ Ambiguous.cljs:28
G__59769__2 @ impl.cljc:64
missionary$impl$Ambiguous$backtrack @ Ambiguous.cljs:34
missionary$impl$Ambiguous$branch @ Ambiguous.cljs:73
missionary$impl$Ambiguous$discard @ Ambiguous.cljs:184
missionary$impl$Ambiguous$done @ Ambiguous.cljs:212
eval @ Ambiguous.cljs:233
missionary$impl$Ambiguous$transfer @ Ambiguous.cljs:237
eval @ Ambiguous.cljs:11
cljs$core$_deref @ core.cljs:688
cljs$core$deref @ core.cljs:1477
eval @ Reduce.cljs:16
eval @ Reduce.cljs:16
missionary$impl$Reduce$transfer @ Reduce.cljs:21
missionary$impl$Reduce$ready @ Reduce.cljs:30
G__56331 @ Reduce.cljs:36
G__57122__1 @ Ambiguous.cljs:334
missionary$impl$Sequential$step @ Sequential.cljs:45
eval @ Sequential.cljs:58
missionary$impl$sleep_cancel @ impl.cljs:201
eval @ impl.cljs:195
missionary$impl$Sequential$kill @ Sequential.cljs:26
eval @ Sequential.cljs:11
missionary$impl$Ambiguous$cancel @ Ambiguous.cljs:92
missionary$impl$Ambiguous$walk @ Ambiguous.cljs:116
missionary$impl$Ambiguous$cancel @ Ambiguous.cljs:102
missionary$impl$Ambiguous$walk @ Ambiguous.cljs:116
missionary$impl$Ambiguous$kill @ Ambiguous.cljs:85
eval @ Ambiguous.cljs:9
eval @ Reduce.cljs:8
missionary$impl$racejoin_cancel @ impl.cljs:153
eval @ impl.cljs:177
eval @ electric_client.cljs:56
01:26:13.954 user.cljs:21 Reactor failure: cljs$core$ExceptionInfo {message: 'Remote error - 1011 …apter$WebSocketPingPongListener$12d400b6 OPEN met', data: {…}, cause: null, name: 'Error', description: undefined, …}cause: nullcolumnNumber: undefineddata: {meta: {…}, cnt: 0, arr: Array(0), __hash: -15128758, cljs$lang$protocol_mask$partition0$: 16647951, …}description: undefinedfileName: undefinedlineNumber: undefinedmessage: "Remote error - 1011 org.eclipse.jetty.websocket.core.exception.WebSocketException: WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN met"name: "Error"number: undefinedstack: "Error: Remote error - 1011 org.eclipse.jetty.websocket.core.exception.WebSocketException: WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN met\n at new cljs$core$ExceptionInfo ()\n at Function.eval [as cljs$core$IFn$_invoke$arity$3] ()\n at Function.eval [as cljs$core$IFn$_invoke$arity$2] ()\n at eval ()\n at hyperfiddle$electric_client$boot_with_retry_$_cr63073_block_10 ()\n at eval ()\n at Function.G__59769__0 [as cljs$core$IFn$_invoke$arity$0] ()\n at eval ()\n at Object.missionary$impl$Sequential$step [as step] ()\n at eval ()"[[Prototype]]: Errorcljs$core$IPrintWithWriter$: {}cljs$core$IPrintWithWriter$_pr_writer$arity$3: ƒ (obj,writer,opts)toString: ƒ ()constructor: ƒ cljs$core$ExceptionInfo(message,data,cause)[[Prototype]]: Object
eval @ user.cljs:21
missionary$impl$Sequential$step @ Sequential.cljs:45
eval @ Sequential.cljs:53
missionary$impl$Sequential$step @ Sequential.cljs:43
eval @ Sequential.cljs:53
missionary$impl$racejoin_terminated @ impl.cljs:163
eval @ impl.cljs:172
eval @ impl.cljs:178
eval @ electric_client.cljs:56
01:26:13.957 websocket.cljs:12 WebSocket connection to '' failed:
s#2023-05-2323:42braai engineerSo symptoms are:
• deploy to Fly — Reactor failure
• local prod build w/optimizations :simple throw Uncaught TypeError: Cannot set properties of null (setting 'prev') on startup & inputs throw this on click or type:
in.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7996 Uncaught Error: No matching clause:
at hyperfiddle.electric_dom2.happen (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7996:487)
at cljs.core.swap_BANG_.cljs$core$IFn$_invoke$arity$3 (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:1575:215)
at Function.f [as cljs$core$IFn$_invoke$arity$1] (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:1531:256)
at HTMLInputElement.<anonymous> (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7995:339)
h
• local dev - everything works, no errors.
• no server logs visible throughout. Is there a way to get more logs?#2023-05-2400:03Dustin Getzonce the first low level NPE has happened the app is crashed, no further errors produced from this undefined state are interesting #2023-05-2400:04Dustin Getzdo you have a git clone url?#2023-05-2400:04braai engineerI’ll DM (it’s a private repo)#2023-05-2400:04Dustin Getzok#2023-05-2323:44braai engineerHas anyone seen an error like this in prod when interacting with UI components, even just clicking on input (focus change)?
Uncaught Error: No matching clause:
at hyperfiddle.electric_dom2.happen (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7996:487)
at cljs.core.swap_BANG_.cljs$core$IFn$_invoke$arity$3 (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:1575:215)
at Function.f [as cljs$core$IFn$_invoke$arity$1] (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:1531:256)
at HTMLInputElement.<anonymous> (main.362B9B6A4E66C2E4112AB2DE5B5C5B41.js:7995:339)
h#2023-05-2323:48braai engineerThis function:
(defn happen [s e]
; Todo, we need a buffer (unbounded) to force a nil in between overlapping events to fix race
; Buffer is unbounded because all events matter. (This is sequential unbounded queue)
(case (:status s)
:idle {:status :impulse :event e} ; rising edge
:pending {:status :impulse :event e} ; supersede the outstanding event with a new event
:impulse (assert false "two events in the same frame? that's weird and wrong")))
I assume (:status s) is nil.#2023-05-2400:50Dustin Getzfor this to happen, iiuc, either the app is already crashed and you interacted again from the crashed state, or you ran into a very subtle electric bug that we call the “when true” bug in which an impossible nil can be observed in situations like (when (not= nil x) (prn x)). i believe the bug is fixed in the next major Electric runtime version that Leo has been working on, we have a workaround for this as well. Please confirm that the app wasn’t already broken when you saw this? #2023-05-2403:50braai engineerHow can I tell if the app has crashed if there are no logs and web requests still get served? Is there maybe a special Electric endpoint that can tell me the state of the reactor?#2023-05-2410:13Dustin Getzjs error#2023-05-2411:17braai engineerDoes electric-xtdb-starter have to be locked to missionary b.27 to work? https://github.com/hyperfiddle/electric-xtdb-starter/blob/master/deps.edn#L14#2023-05-2411:29Geoffrey Gaillard@U053XQP4S#2023-05-2411:31leonoelit should work with b.30#2023-05-2411:34braai engineerUpdate: fixed by moving user.clj. See thread.
Can’t get :prod build to work - even locally. :dev works fine but prod throws this JS exception:
Uncaught ReferenceError: $tcoll is not defined
at cljs.core.TransientHashMap.cljs$core$ITransientAssociative$_assoc_BANG_$arity$3 (main.E2C739D1501A78B5E69CCC0330A5910B.js:2106:374)
at cljs.core._assoc_BANG_ (main.E2C739D1501A78B5E69CCC0330A5910B.js:991:119)
at cljs.core.PersistentHashMap.fromArrays (main.E2C739D1501A78B5E69CCC0330A5910B.js:2097:286)
at main.E2C739D1501A78B5E69CCC0330A5910B.js:3544:69
at main.E2C739D1501A78B5E69CCC0330A5910B.js:25424:4
c
There is a hint of this during the shadow build:
------ WARNING #1 - -----------------------------------------------------------
File: ~/.m2/repository/org/clojure/clojurescript/1.11.60/clojurescript-1.11.60.jar!/cljs/core.cljs:8266:36
--------------------------------------------------------------------------------
8263 | (-persistent! [tcoll] (.persistent! tcoll))
8264 |
8265 | ITransientAssociative
8266 | (-assoc! [tcoll key val] (.assoc! tcoll key val))
------------------------------------------^-------------------------------------
variable $tcoll is undeclared
--------------------------------------------------------------------------------
8267 |
8268 | ITransientMap
8269 | (-dissoc! [tcoll key] (.without! tcoll key))
8270 |
--------------------------------------------------------------------------------
Could this be a shadow-cljs bug?#2023-05-2411:55xificurCyou should apply what I'd call comment-bisecting. Comment out half of your application and rebuild. Keep repeating until you narrow down on a specific issue. At that point it's easier to point fingers#2023-05-2411:56xificurCThe worst I hit this year was a google closure optimization bug#2023-05-2411:57braai engineerThis happens even with :optimizations :none during uberjar build (using shadow/compile instead of shadow/release). Not sure if that applies here?#2023-05-2411:58xificurCsure, probably not a GCC bug in this case, but the bisect should still help#2023-05-2423:36braai engineerFixed by moving user.clj to src-dev/user.clj and using :extra-paths.
The user namespace that prevented the logging .class from being compiled. It’s a footgun: https://www.karimarttila.fi/clojure/2023/03/14/creating-uberjar-for-clojure-fullstack-app.html#2023-05-2413:30braai engineerUpdate: fixed (see below).
I’m running a pure :dev build on a “prod” machine because I’m blocked on :prod build issues.
Everything works but after a while, the connection drops with this in the server logs:
ERROR hyperfiddle.electric-jetty-adapter: Websocket error
java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__78555.invoke(electric_jetty_adapter.clj:76)
at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__78396.invoke(websocket.clj:157)
Is this expected? It can’t seem to reconnect after dropping (average Internet here in ZA).
Is this NPE the cause of reactor breaking, or the symptom of a broken reactor?#2023-05-2414:03braai engineerUpdate: fixed by moving user.clj to src-dev/user.clj and adding "src-dev" to :extra-paths of :dev build alias. user namespace is footgun that prevents logging .class from being built AOT: https://www.karimarttila.fi/clojure/2023/03/14/creating-uberjar-for-clojure-fullstack-app.html
---
OK, I’ve managed to reproduce the Reactor failure using electric-xtdb-starter repo with minimum changes to get a uberjar out via src-build/build.clj and :optimizations :none. Repo: https://github.com/theronic/electric-xtdb-starter (soz I pushed to master branch)
I tried with both Jetty 9 & 10. With Jetty 10 I don’t see any logs, but 9 shows this in server logs:
WARN org.eclipse.jetty.websocket.common.WebSocketSession: Exception while notifying onClose
java.lang.NoClassDefFoundError: clojure.tools.logging.impl.LoggerFactory
at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_close__14039.invoke(electric_jetty_adapter.clj:66)
at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__13906.invoke(websocket.clj:159)
at ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6.onWebSocketClose(Unknown Source)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onClose(JettyListenerEventDriver.java:149)
at org.eclipse.jetty.websocket.common.WebSocketSession.callApplicationOnClose(WebSocketSession.java:394)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.close(AbstractWebSocketConnection.java:225)
at org.eclipse.jetty.websocket.common.WebSocketSession.close(WebSocketSession.java:130)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.openSession(AbstractEventDriver.java:221)
at org.eclipse.jetty.websocket.common.WebSocketSession.open(WebSocketSession.java:493)
at org.eclipse.jetty.websocket.common.WebSocketSession.onOpened(WebSocketSession.java:459)
at org.eclipse.jetty.io.AbstractConnection.onOpened(AbstractConnection.java:213)
...
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
at java.base/java.lang.Thread.run(Thread.java:858)
Caused by: java.lang.ClassNotFoundException: clojure.tools.logging.impl.LoggerFactory
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:827)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1096)
... 28 common frames omitted
Steps to repro:
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty`
clojure -X:build uberjar :jar-name "app.jar" :version '"'$HYPERFIDDLE_ELECTRIC_APP_VERSION'"' :optimizations :none
java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -jar app.jar
Same results for Missionary b.30.#2023-05-2414:17Geoffrey GaillardCaused by: java.lang.ClassNotFoundException: clojure.tools.logging.impl.LoggerFactory
It looks like you don’t have the same logging deps between dev and prod.
You might want to look at clojure.tools.logging documentation and configure your logger.#2023-05-2414:24braai engineerWhere am I overriding logging deps in prod?
deps are from here: https://github.com/hyperfiddle/electric-xtdb-starter/blob/master/deps.edn#2023-05-2414:33Geoffrey GaillardThis might be missing from your deps.edn
https://github.com/hyperfiddle/electric-xtdb-starter/blob/ffe3ed23cc51e7dd7a001263b52d52a6fd00738a/deps.edn#LL35C13-L35C87#2023-05-2414:38braai engineerhttps://github.com/theronic/electric-xtdb-starter/blob/master/deps.edn#L40 ?#2023-05-2415:08Geoffrey GaillardOk this is not it, sorry.
Still the issue is related to a clojure.tools.logging misconfiguration.
Maybe a dependency conflict?#2023-05-2415:20Geoffrey GaillardLooks like an issue with the uberjar.
This works in the meantime:
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty` clojure -J-DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -M -m prod
#2023-05-2415:28Geoffrey GaillardAfter running it you’ll get No implementation of method: :latest-completed-tx of protocol: #'xtdb.api/PXtdb found for class: clojure.lang.Var$Unbound
~It’s because ~ sequentially. In clojure, binding binds in parrallel, so you want this~
Edit: I missread. The reason is user.clj is not loaded#2023-05-2416:33braai engineerAha, it’s because in the prod main, !xtdb is not set like in the dev main:
(defn main [& args]
(println "Starting XTDB...")
(alter-var-root #'!xtdb (constantly (start-xtdb!))) ...
This is probably the reason why it’s error’ing out in my repo too 🙂
Now to fix the logging issue…#2023-05-2416:43braai engineer@U2DART3HA OK so I have to use -J flag to set jvm property. ~hmm, I’m getting this error despite setting the env vars:~
~(I’m not familiar with -J-D syntax)~#2023-05-2423:36braai engineerFixed by moving user.clj to src-dev/user.clj and using :extra-paths.
The user namespace that prevented the logging .class from being compiled. It’s a footgun: https://www.karimarttila.fi/clojure/2023/03/14/creating-uberjar-for-clojure-fullstack-app.html#2023-05-2416:31braai engineerHas anyone managed to get a working :prod (uberjar) going with the Electric XTDB starter repo?#2023-05-2509:46Geoffrey GaillardAdding xtdb.api to the :ns-compile list https://github.com/theronic/electric-xtdb-starter/blob/eebb6d16880551315f011f945667675c0666c036/src-build/build.clj#L56 seems to help.
:ns-compile [prod xtdb.api]#2023-05-2511:35braai engineerThanks, will try that too 🙂#2023-05-2421:11braai engineerUnder the Deployment heading in the electric-starter-app README, it states to run build-client, then uberjar step, but the uberjar step includes build-client. So many ppl trying out Electric will be building client twice (I did this for a long time until I read build.clj).
Maybe can move build-client to its own code block? https://github.com/hyperfiddle/electric-starter-app#deployment#2023-05-2516:22Dustin GetzThis is a readme-only fix right?#2023-05-2517:01braai engineerYes, assuming there is no another reason to run build-client separately (maybe in parallel).#2023-05-2716:20Dustin Getzthis is done, thanks#2023-05-2513:19jmckitrickI’m trying to internalize the use of new in electric. It’s not quite clicking with me when it should be used, and why, if everything in clojure is ostensibly about functions, it’s not just a function call?#2023-05-2513:21Dustin Getzfor the same reason reagent uses square brackets. we could have chosen square brackets.#2023-05-2513:22jmckitrickI remember seeing that in the docs, but this isn’t a compenent, right?#2023-05-2513:22Dustin Getzclojure is dynamically typed so there is no robust way to distinguish between reactive fn call and regular function call, and since Electric prioritizes interop with regular clojure, you need a way to indicate the call convention #2023-05-2513:22jmckitrickdb (new (db/latest-db> user/!xtdb))
#2023-05-2513:22Dustin Getzah that is the second usage of new, which is to await a missionary flow#2023-05-2513:23jmckitrickSounds like I need to do a little reading on missionary#2023-05-2513:24Dustin Getzawaiting a missionary flow is like awaiting a promise, except instead of a promise to 1 async future value, a flow is a sequence (possibly infinite) of async future values#2023-05-2513:25jmckitrickThat’s the streams and signals part, right?#2023-05-2513:27Dustin Getzyes, streams and signals are kinds of flows#2023-05-2513:40xificurCyou only need to learn missionary if you're going to write integrations at the edge of the system (e.g. bridging a database to electric, a third-party js API that returns promises, ...). Electric largely abstracts missionary away. If you need help integrating you can ask us here and we'll see if we can help#2023-05-2513:41jmckitrickcool, thanks.#2023-05-2513:42jmckitrickWe’re currently working on an in-house app for sprint planning poker, so we can get a feel for it#2023-05-2514:21jjttjjI'm trying to render markdown text in electric. I'm using https://github.com/nextjournal/markdown which works in cljc and I'm not really picky if the parsing occurs on the client or server.
I've tried the below approaches and a few others but not having any luck. At a high level is there an obvious approach to this? Is just parsing the markdown to an html string and setting innerHTML of the dom/node the way to go here?
(Edit: I think for the markdown case here it probably is good to just use innerHTML or maybe just use a js library to handle all the parsing after the text hits the dom, but I am sort of curious about the way in electric to think about a transformation of data to nodes)
(defmacro electric-markdown1 [text]
(let [data (md/parse text)]
(walk/postwalk
(fn [x]
(if (vector? x)
(let [[tag & content] x]
(case tag
:<> `(dom/text ~@content)
(list* (symbol "hyperfiddle.electric-dom2" (name tag)) content)))
x))
(md.transform/->hiccup data))))
;; Can't take value of macro
(e/defn hiccup-to-dom
[hiccup-data]
(let [tag (name (first hiccup-data))
children (if (map? (second hiccup-data))
(drop 2 hiccup-data)
(drop 1 hiccup-data))]
(println "tag" tag)
(dom/element tag
(map (fn [child]
(if (string? child)
(dom/text child)
(hiccup-to-dom child)))
children))))
(e/defn electric-markdown3 [text]
(let [data (md/parse text)
hiccup (md.transform/->hiccup data)]
(hiccup-to-dom. hiccup)))
;;; stack overflow during compilation#2023-05-2515:00xificurC• I don't see any usage of electric-markdown1
• in hiccup-to-dom you have to use e/for instead of map on the children. map is clojure land so you cannot call electric code like dom/text in it
• electric-markdown3 looks alright except it's calling the broken hiccup-to-dom #2023-05-2515:02xificurCre data to nodes, that sounds like an interpreter, which hiccup-to-dom is#2023-05-2515:03xificurCthe advantage of feeding data into electric instead of a raw string is you'll get fine-grained reactive DOM updates as the interpreter walks the data. Unchanged data nodes, provided they are properly keyed, will leave the DOM untouched#2023-05-2515:04xificurCthere's also an overhead to running electric, so it also might be that setting innerHTML will be faster#2023-05-2515:06Dustin GetzOur tutorial app has a working example - https://github.com/hyperfiddle/electric-examples-app/blob/60e2e6f8c34c6397da91af7d067bd977a75240c1/src/user_main.cljc#L137-L145C47#2023-05-2515:09jjttjj> • electric-markdown3 looks alright except it's calling the broken hiccup-to-dom
Trying this give me:
(e/defn hiccup-to-dom [hiccup-data]
(let [tag (name (first hiccup-data))
children (if (map? (second hiccup-data))
(drop 2 hiccup-data)
(drop 1 hiccup-data))]
(dom/element tag
(e/for [child children]
(if (string? child)
(dom/text child)
(hiccup-to-dom. child))
children))))
(e/defn electric-markdown3 [text]
(let [data (md/parse text)
hiccup (md.transform/->hiccup data)]
(hiccup-to-dom. hiccup)))
...
(e/client
(electric-markdown3. "#hi"))
Encountered error when macroexpanding hyperfiddle.electric/boot.
Failed to analyse form
{:in [(map? (second hiccup-data))]}
`
> Our tutorial app has a working example
Thanks yea I have the innerHTML method working fine
(edit: edited this code a bit to add a . to the recursive hiccup-to-dom call. still getting compilation errors, will report back if something is meaningfully different/promising though)#2023-05-2515:43xificurCah, recursive self calls not implemented yet, https://github.com/hyperfiddle/electric/blob/15fb0219ff0d4daf507bd513750c698502700d2a/src-docs/user/electric/electric_recursion.cljc for a workaround#2023-05-2515:48jjttjjI was just looking at that! I think that's getting me closer#2023-05-2516:02jjttjjOk yeah I think this works, using code from dom2/element. Thanks for the help!
(e/def h2e)
(e/defn electric-markdown3 [text]
(let [data (md/parse text)
hiccup (md.transform/->hiccup data)]
(binding [h2e (e/fn [hiccup-data]
(let [tag (first hiccup-data)
children (if (map? (second hiccup-data))
(drop 2 hiccup-data)
(drop 1 hiccup-data))]
(dom/with (dom/new-node dom/node (name tag))
;; hack: speed up streamy unmount by removing from layout first
;; it also feels faster visually
(e/on-unmount #(set! (.. node -style -display) "none")) ; hack
(e/for [child children]
(if (string? child)
(dom/text child)
(h2e. child))))))]
(h2e. hiccup))))#2023-05-2516:20Dustin Getzfwiw the streamy unmount lag might be fixed in the next version, we have a pretty good idea of what is happening, it was a mistake in how unmounts are synchronized over network#2023-05-2517:06jjttjjHere's a bit of a refactored version if anyone's interested in the future. supports :div.classes.like.this and optional attriburtes, but not :div#id yet. (had enough of yak shaving on this for today)
(e/def H2e)
(e/defn ElectricMarkdown [text]
(let [data (md/parse text)
hiccup (md.transform/->hiccup data)]
(binding [H2e (e/fn [hiccup-data]
(let [[tag & tagclasses] (str/split (name (first hiccup-data)) #"\.")
[children attrs] (if (map? (second hiccup-data))
[(second hiccup-data) (drop 2 hiccup-data)]
[(drop 1 hiccup-data)])
attrs (cond-> attrs
(seq tagclasses) (update :class concat tagclasses))]
(dom/with (dom/new-node dom/node tag)
(when (seq attrs)
(dom/props attrs))
(e/on-unmount #(set! (.. dom/node -style -display) "none")) ; hack
(e/for [child children]
(if (string? child)
(dom/text child)
(H2e. child))))))]
(H2e. hiccup))))
(comment
(ElectricMarkdown. "# Hello"))#2023-05-2518:55xificurCYou could write a transformer for that library, skip the middleman#2023-05-2717:31Dustin GetzI'm considering making the default electric-starter-app use Datomic (now that it's free) instead of Datascript. I considered XTDB but people here are having trouble configuring it and we don't want to be first-line support for XTDB matters. Any thoughts?#2023-05-2808:42teodorluWhich Datomic backend are you planning to use? Postgres? Sqlite? In memory?#2023-05-2811:28Dustin Getzin memory for now, we also might move the starter app off fly as well to avoid putting a cross region database setup into the starter app#2023-05-2813:23Daniel JompheDatomic is sure closer to what we’d use for durable storage, so this could improve the starter app’s usefulness.
What might be the most useful to this community in this change is how it may help newcomers understand how true it is that the DB is your backend’s DB, not what people usually use Datascript for, a frontend cache.#2023-05-2813:24Daniel JompheApparently Datomic Cloud dev-local (used in memory or on filesystem storage) will be published to a public maven repo in the next weeks, BTW. But I suppose your target is the Peer API of Datomic non-cloud.#2023-05-2814:08Dustin GetzThe Datomic Client API (which is the only API available in Cloud) requires a deeper Electric integration (to adapt the core.async API). We have a POC of this adapter (used by the datomic browser demo) but it needs polish, and to truly polish it we need to partner with someone using Cloud in production who can give us real world feedback#2023-05-2911:38Norman KabirHave you considered Datalevin? It's open source and has both embedded and client/server configurations.
https://github.com/juji-io/datalevin#2023-05-2913:57tatutXTDB is super simple imo to get started with, just a single function call to start an in-memory node. Is the eventual production grade setup necessary for a starter example?#2023-05-2914:15kishimaWill I get tomatoes if I ask why not Postgres? 👀#2023-05-2914:31braai engineer@U3X7174KS I would choose the storage option that is easiest to deploy for a real-world but not necessarily “Web Scale” app. And that is probably Postgres. Dynamo great but AWS-specific. I would say Sqlite but where to put the file inbetween app starts? Most platforms offer a simple free Postgres DB.
@U09K620SG Tesserae uses Datalevin. Great for embedded DB if you don’t need time travel. I suspect “real-world” Electric adopters want time travel though, but poll.
If I were you, I’d go with SQL-based Datomic in such a way that it is easy to deploy at “some scale,” i.e. two transactors and people can migrate to Cloud later.#2023-05-2914:32braai engineerIssue with Sqlite is that it’s file-based because aimed at embedded, which means you can’t have multiple writers for failover (maybe this has changed and I’m wrong about this).#2023-05-3003:09AmosHi, I’m looking into the electric starter app. I wonder when exactly the shadow load the app.todo-list. Thanks in advance.#2023-05-3008:07xificurCHi, do you mean https://github.com/hyperfiddle/electric-starter-app/blob/cd8e92b8aa0de51f02d59036cfc02fc6dc9df650/src/user.cljs#L10?#2023-05-3013:16AmosNo, I mean about how to call https://github.com/hyperfiddle/electric-starter-app/blob/cd8e92b8aa0de51f02d59036cfc02fc6dc9df650/src/user.cljs#LL14C1-L14C42 when execute the code.#2023-05-3015:12xificurCshadow-cljs https://shadow-cljs.github.io/docs/UsersGuide.html#_lifecycle_hooks for hot code reloading. The app entrypoint is set in https://github.com/hyperfiddle/electric-starter-app/blob/cd8e92b8aa0de51f02d59036cfc02fc6dc9df650/shadow-cljs.edn#L11-L12#2023-05-3100:13AmosThank you. That helps a lot.#2023-05-3006:25noonianHello! I am seeing behavior that I don't understand and I am wondering if it could be a bug in electric and/or e/for. In this snippet, my expectation is that no code inside a when block would ever execute when the value of the condition is false, but I see that happen with the code below. I'd appreciate any insight into what is happening here. I am using electric v2-alpha-284-g0bc5f4bc and missionary b.30
(e/defn ComponentWithFor [items]
(dom/ul
(e/for [item items]
(dom/li (dom/text item)))))
(e/defn StrangeBugRepro []
(let [!x (atom 0)
x (e/watch !x)]
;; When this button is clicked, "x: nil" prints to the console
(ui4/button (e/fn []
(reset! !x nil))
(dom/text "Reset x to nil"))
(when x
;; This should never print if x is nil... right? It does if...
(println "x:" x)
;; ...there is another electric function call in the
;; when that uses e/for and the e/for depends on x. Does not
;; occur if e/for is inlined.
(ComponentWithFor. (range x))
)))#2023-05-3008:40xificurCHey, thanks for the report and the repro! We're aware of a similar issue but it includes a server transfer, while your case involves only involves an e/for. I logged a ticket for this. The internals are going through a major rewrite that might solve this issue, otherwise we'll look into it after the internal rewrite lands.
Do you need help finding a workaround for your project?
Including a minimal RCF for reference.#2023-05-3013:25braai engineer@U052PH695 does the same thing happens when not using the ui4/button helper? E.g. (dom/a (dom/on "click" (e/fn [e] (reset! !x nil) …))
I’ve had similar issues with ui4/button, but may be a different edge case.#2023-05-3017:22noonianThank you, it's helpful to know this is unintended behavior 🙂 I'm excited for the rewrite, and any workarounds in the meantime would be helpful but I can probably live with it for the time being. I'm also happy to help test the new implementation once it's ready for that#2023-05-3114:49AmosHi, I wonder what is the answer for Quiz: in (e/def x (e/server (e/watch !x))), why do we e/watch on the server? Also, is there API docs for the electric?#2023-05-3118:57xificurCA server atom can only be watched on the server and vice versa . The client can read the watched reactive values, provided they are serializable#2023-05-3118:58xificurCno API docs yet, sorry! We have a https://electric-examples-app.fly.dev/ that should help you ramp up quickly#2023-05-3119:42Dustin Getzto add to what Peter wrote - the atom !x is not serializable and cannot transfer; the watch is implemented by calling clojure.core/add-watch on the atom to register a change callback, therefore the watch must co-locate with the atom#2023-06-0101:23AmosFor this line #?(:clj (defonce !x (atom true))) , how can we know that the !x is a ‘server’ atom? not perhaps ‘client’ atom? Perhaps if not specified, will be the server one?#2023-06-0101:28tobiasThe #?(:clj ...) reader conditional means that the defonce is only executed in clojure (i.e. the server), so that atom only exists on the server. If you wanted a client-side atom you could use #?(:cljs (defonce !x (atom true))) . Docs for reader conditionals: https://clojure.org/guides/reader_conditionals#2023-06-0102:06AmosGot it. Thank you so much.#2023-06-0109:17AmosHi, I’m looking into the xtdb starter code. I wonder why we have to define a latest db like https://github.com/hyperfiddle/electric-xtdb-starter/blob/ffe3ed23cc51e7dd7a001263b52d52a6fd00738a/src/app/todo_list.cljc#L69, since the xtdb tutorial doesn’t do this.#2023-06-0110:16Dustin Getzit’s reactive, that function is subscribing to the stream of tx changes and using that to automatically refresh the db val#2023-06-0113:11braai engineerAny known issues with slow Internet causing Electric Websocket to drop on timeout and render a blank screen? Requires reload to recover:
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info]ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info]clojure.lang.ExceptionInfo: Websocket pong timeout.
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at hyperfiddle.electric_jetty_adapter$make_heartbeat$cr7506_block_4__7516.invoke(electric_jetty_adapter.clj:22)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at cloroutine.impl$coroutine$fn__5168.invoke(impl.cljc:60)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.Sequential.step(Sequential.java:88)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.Sequential$1.invoke(Sequential.java:112)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.core$absolve$fn__5532$fn__5533.invoke(core.cljc:141)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.RaceJoin.terminated(RaceJoin.java:51)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.RaceJoin$1.invoke(RaceJoin.java:71)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.RaceJoin$2.invoke(RaceJoin.java:86)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.Sleep.trigger(Sleep.java:63)
2023-06-01T11:07:44Z app[3d8d1ddbe43e89] jnb [info] at missionary.impl.Sleep$Scheduler.run(Sleep.java:35)
#2023-06-0114:43braai engineerHmmk, blank screen issue might just be OOM.#2023-06-0118:26Dustin GetzYeah this is the 60 second timeout, if a heartbeat is missed for 60 seconds the connection is dropped#2023-06-0118:26Dustin GetzI believe the starter app is configured to auto-reconnect#2023-06-0118:27Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md has a screenshot of the auto-reconnect#2023-06-0118:30Dustin GetzYes, the page will teardown (turn white) on disconnect (losing state), and then boot again on reconnect. In the future, it won't lose client state, rather it will preserve the state and gracefully reconnect as if nothing had happened – this feature is blocked on the big refactor that Leo has been working on#2023-06-0118:34Dustin GetzHowever making an interactive app with forms and such work gracefully despite intermittent connectivity is a tall order, essentially asking for offline-first software. This is something we're thinking a lot about for electric-ui5 which we've been designing, however I can't promise we will solve the problem of offline-first as that's not actually the problem we're trying to solve, we're actually working on optimistic updates which is closely related to offline first but not quite the same#2023-06-0209:42braai engineerThx. Possible to throw an Exception like Pending when Websocket is down so I can catch it and disable things on UI, but preserve read-only state? E.g. (try … (catch Offline ex …)) ? I’m okay with Electric throwing an exception if it can’t run a Fn that happens on the server.#2023-09-1918:27Dave MaysIf offline-first worked with Electric it feels like it would be the best of all worlds. I know the focus is internal web apps, but offline first opens the doors to all kinds of things eventually like mobile apps.
Im curious though, would this involve somehow getting XTDB (or some other DB) to work in the client as well as the back end, and sync between them? I'd love an offline-first capable datalog DB, and haven't found anything except for those below. Is a client-side XTDB part of what would be required for optimistic updates?
https://www.instantdb.com/ (Datahike seems to have left the client side for now, and Datascript only seemed maybe made for this?)#2023-06-1602:48Vincentcool idea#2023-06-1602:49Vincentyeah radio buttons are a nightmare lol i was thinking maybe i can avoid this x)#2023-06-1610:38s-olI think you can get the same behaviour quite easily with multiple checkboxes (that can be styled to communicate exclusivity) as well#2023-06-1616:45Vincent@U05AL1ZH8TW that occurred to me when looking at the todo list example again, i think this is a good idea too#2023-06-1713:55AmosHi, I have look into the https://electric.hyperfiddle.net/user.demo-reagent-interop!ReagentInterop. I wonder whether can we use react hook inside the reagent component?#2023-06-1714:13Dustin GetzYes you can (assuming Reagent supports it, I am not a reagent user anymore), did you try?#2023-06-1714:13Dustin GetzA seamless interop between electric/reagent/react should be possible#2023-06-1714:35Dustin Getz[From @U0577BSF2CC by DM]
Hi, I have try to follow the reagent interop the following is what I try to do:
(defn block [initialValue]
#?(:cljs
(let [[editor] (react/useState (fn [] (withReact (createEditor))))]
[:> Slate {:editor editor :initialValue (clj->js initialValue)}])))
However, it shows that the hook can’t be at the outside of function component. Does this interop support the react hook?#2023-06-1714:36Dustin GetzDoes this code work in Reagent?#2023-06-1714:36Dustin GetzIf you have a link to Reagent docs explaining this - please send#2023-06-1716:59VincentThe latest and greatest in savage workflows:
don't know how to do spotify web playback sdk in electric, or cljs, so
1. get it working in javascript
2. ask the good chap (chatGPT4) to clojurescript it! (turns it into cljs no prob)
3. go from cljs to electric (still got some questions here but learning! 😅 )#2023-06-1717:26Dustin Getzyou can interop directly to javascript from clojurescript#2023-06-1717:26Dustin Getzget a .js file on the page that exposes a few functions, then call them from electric#2023-06-1717:27Dustin Getzexample - (js/alert "hi")#2023-06-1717:42Dustin Getzalso if you have working cljs from chatgpt, just drop it in an ordinary defn and call it from Electric#2023-06-1722:07VincentMaybe that is a cleaner way to go. I actually ended up with the following:
#?(:cljs (defn play [spotify_uri]
(PUT (str "" @device-id)
{:with-credentials? false
:headers {"Authorization" (str "Bearer " (get-token))
"Content-Type" "application/json"
"Accept" "application/json"}
:query-params {"device_id" @device-id}
:body (js/JSON.stringify (clj->js {:uris [spotify_uri]}))
:handler (fn [response]
(.log js/console "Playback started"))
:error-handler (fn [response]
(.log js/console "Error in playback"))})))
#?(:cljs (defn on-spotify-web-playback-sdk-ready []
(let [player (js/Spotify.Player. (clj->js {:name "Web Playback SDK Quick Start Player"
:getOAuthToken (fn [cb] (cb (get-token)))
:volume 0.5}))]
(.addListener player "ready" (fn [in] ;{:keys [device_id]} :as
(let [jin (js->clj in :keywordize-keys true)]
(reset! device-id (get-in jin [:device_id]))
(.log js/console "!#: " in)
(.log js/console "!#: " @device-id)
(.log js/console "!#: " jin))))
(.connect player))))
(e/defn SpotifyPlayer []
(e/client
(set! js/window.onSpotifyWebPlaybackSDKReady on-spotify-web-playback-sdk-ready)
;(.log js/console "addeen spoteefai playeur")
(dom/div
(ui/button (e/fn [] (e/client (play "
which I like because I have it rigged up to the ui/button and can sorta model an interface around that. What do you think#2023-06-1812:09Dustin Getzi think that's about the best you can do with the current ui4/button#2023-06-1904:54Vincentworks great 😅#2023-06-1722:07VincentMaybe that is a cleaner way to go. I actually ended up with the following:
#?(:cljs (defn play [spotify_uri]
(PUT (str "" @device-id)
{:with-credentials? false
:headers {"Authorization" (str "Bearer " (get-token))
"Content-Type" "application/json"
"Accept" "application/json"}
:query-params {"device_id" @device-id}
:body (js/JSON.stringify (clj->js {:uris [spotify_uri]}))
:handler (fn [response]
(.log js/console "Playback started"))
:error-handler (fn [response]
(.log js/console "Error in playback"))})))
#?(:cljs (defn on-spotify-web-playback-sdk-ready []
(let [player (js/Spotify.Player. (clj->js {:name "Web Playback SDK Quick Start Player"
:getOAuthToken (fn [cb] (cb (get-token)))
:volume 0.5}))]
(.addListener player "ready" (fn [in] ;{:keys [device_id]} :as
(let [jin (js->clj in :keywordize-keys true)]
(reset! device-id (get-in jin [:device_id]))
(.log js/console "!#: " in)
(.log js/console "!#: " @device-id)
(.log js/console "!#: " jin))))
(.connect player))))
(e/defn SpotifyPlayer []
(e/client
(set! js/window.onSpotifyWebPlaybackSDKReady on-spotify-web-playback-sdk-ready)
;(.log js/console "addeen spoteefai playeur")
(dom/div
(ui/button (e/fn [] (e/client (play "
which I like because I have it rigged up to the ui/button and can sorta model an interface around that. What do you think#2023-06-1813:43AmosHi, I try to use a react component in electric framework:
<Slate
editor={editor}
initialValue={initialValue}
onChange={value => {
const isAstChange = editor.operations.some(
op => 'set_selection' !== op.type
)
if (isAstChange) {
// Save the value to Local Storage.
const content = JSON.stringify(value)
localStorage.setItem('content', content)
}
}}
>
The onChange part is where I should send the data to the backend from frontend. However, I can’t use e/server inside the function since it is not define as e/defn. Does any one know how to fix this? I think this is a common question when we try to use react component from other library.#2023-06-1814:13Dustin GetzYou'll need to use callback props to get values out of react using react idioms#2023-06-1814:13Dustin GetzWe don't have an easy example of how to do this yet, sorry#2023-06-1814:14Dustin GetzYou can look at our https://github.com/hyperfiddle/electric/blob/585dd245e4e406940c16a9fdacfa4db79ba24bfd/src/contrib/electric_codemirror.cljc#L70-L91, but it's complicated (because the codemirror api is spaghetti)#2023-06-1814:30AmosThank you. I will check that.#2023-06-1915:22s-olI'm not familiar with shadow, is there some meta annotation I can apply to a namespace that is only required on the JVM side to make electric reload that namespace automatically?#2023-06-1915:24s-olright now I can force a reload with (require 'ns :reload) and then trigger a rebuild by saving a file that is shared. Or I could put the namespace into a .cljc and just have the whole file in a jvm-only reader macro but that seems silly also#2023-06-1915:52Dustin Getzim a bit confused can you try to state the root problem in a context-free way#2023-06-1915:52Dustin Getz"Hot code reloading doesn't work the way i expected when i __"#2023-06-1915:54Dustin GetzI think you're saying: Electric reloads changed namespaces when I save but only in CLJC files, is there a way have electric load .CLJ files also?#2023-06-1915:55Dustin GetzDon't you want to reload your ordinary clojure code using the standard Clojure practices, i.e. jack in the repl and send buffer to repl?#2023-06-1918:38s-olI don't really have a clojure background and use an obscure editor so I don't have "send selection/block to REPL" set up currently, so I rather reload the whole namespace and run stuff "manually"#2023-06-1918:40s-olbut for that workflow manual reloading is fine; if I run a snippet manually I can also reload manually. Where it feels "unnatural" is when electric code runs regular-clojure-defn functions, but changing those doesn't trigger an electric reload like changing code in an e/server block does#2023-06-1918:40s-ol(and yeah, you got it right above)#2023-06-2021:01Dustin GetzThe tools.namespace clojure library may help, but it is clumsy to use#2023-06-2021:03Dustin GetzThe best answer is probably to write all your code in CLJC, note that you don't need top level reader macros because presumably most of the code is portable despite you only needing to run it on the JVM. Only java interop would need to be guarded, and only if the file is in the dependency graph originating from a CLJS module#2023-06-2206:58s-olI think I need the reader macros, just because some dependencies are JVM-only and missing namespace references in clojure fns are compiler errors otherwise iirc#2023-06-2206:59s-olbut yeah, it's a reasonable fix and also makes it easy to add client code in that namespace at any point if necessary#2023-06-2207:00s-olOn that note, does the compiler strip unused functions from the client build? In electric functions I'm assuming the server portion is removed, but what about clojure functions defined in imported namespaces?#2023-06-2207:04xificurCthe electric compiler today starts from the entrypoint and does a full program analysis. This means only electric code that is used from the entrypoint gets compiled to cljs code. Does that answer your question?#2023-06-2207:12s-ol@U09FL65DK I was wondering about non-electric code referenced from electric code as well, since @U09K620SG mentioned there's no need to put reader macros if the code compiles on both platforms#2023-06-2207:17xificurCfor non-electric code standard cljs or shadow compilation rules apply#2023-06-2207:23xificurCI think the "no need to put reader macros if code compiles on both platforms" is just to ease the burden of annotating all of your code. If there's no JVM specific calls you can elide the conditional#2023-06-2210:30Dustin Getzclojurescript compiler performs dead code elimination in release builds to reduce artifact size sent over the wire; clojure jar files are not sent over the wire at runtime so dead code is essentially harmless#2023-06-2115:52KeinWhat are the conditions that a e/def or e/defn be re-evaluated?
• [1] when the e/def or e/defn depend on some (e/watch x!) , x! must be a clojure atom, and x! change
• [2] when the e/def or e/defn depend on some e/electric-defined-varible x, and x change
Is it either [1] or [2] ? Please correct me if there’s something not precise.#2023-06-2116:31Dustin GetzIn an Electric context, an expression (f a b) will re-evaluate when any of its parameters have changed#2023-06-2116:35Dustin GetzThe source of a change typically originates from either e/watch on an atom, or m/observe which encapsulates a callback (e.g. a dom event). Note that e/watch can be implemented in terms of m/observe as it's all just callbacks and side effects at the bottom#2023-06-2116:36Dustin GetzOnce a callback has fired, the change will propagate through the DAG, re-evaluating any expression with a dirty parameter#2023-06-2116:37Dustin GetzAll this to say: [1] and [2] are the same thing#2023-06-2116:42Dustin GetzDo you have a specific example that is doing something unintuitive? Electric also has exceptions, particularly Pending , an expr (f a b) will NOT re-evaluate if a parameter becomes Pending, though it will re-evaluate once the exception transitions to a non-exceptional value#2023-06-2116:01KeinI’m trying to fit Clojure’s evaluation model with electric’s streaming evaluation model together, and have some confusion.#2023-06-2117:05Dustin GetzIn summary, Electric is intended to be "correct" in that it will evaluate exactly the minimum necessary, no more no less. (As opposed to say React.js which re-evaluates at the granularity of components but not at the granularity of expressions.) I think you can work backwards from that what the evaluation model has to be, if you accept the leap of faith#2023-06-2117:09Dustin GetzElectric bugfix release - v2-alpha-349 — https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md
• fixes regressions in the dom props/class handling
• fixes e/offload causing over-rendering, we officially bless the use of e/offload now to move a blocking function to a threadpool (e.g. a query). We intend to encourage this pattern in the tutorials and starter apps now. (Previously we had removed e/offload from the tutorials due to this issue)#2023-06-2119:33noonianI believe I found a minor regression with how classes are handled in the new release. I opened https://github.com/hyperfiddle/electric/pull/45 to address the issue#2023-06-2119:26VincentHi I wondering about making a <form> element with multiple <inputS>
like a user account creation form, for example,
am thinking maybe can just stay in electric entirely and hook a POST event to a click#2023-06-2120:25Dustin GetzThe todo app shows one way - https://electric.hyperfiddle.net/user.demo-todos-simple!TodoList#2023-06-2120:28Dustin GetzYou can use the hyperfiddle.electric-ui4 namespace like this, to collect data entry into an atom#2023-06-2120:30Dustin Getzthen use a ui4/button to send it to the server (or use an HTML form with a post http endpoint, but then you are leaving electric)#2023-06-2120:31Dustin Getzthe "UI5" project we've been talking about is an electric-native crud/form solution with lots of bells and whistles, but it's not close to launching yet#2023-06-2122:58Vincentokay great! thanks for the info. makes sense. i am glad there is a bundled ui4 thing to do the atom-save and then the send. ty#2023-06-2122:58Vincentand gl on ui5#2023-06-2207:56s-olI've noticed that when I change my reactive database from within a (ui/button (e/fn [] ...)) in some cases that e/fn updates twice or even causes an infinite update loop. How can I prevent that and "freeze" reactive vars on mount in the e/fn?#2023-06-2208:04xificurCcan you share the code that behaves this way?#2023-06-2208:05s-olI can share the repo, but you probably won't be able to get it to repro easily because I don't have code to seed the database#2023-06-2208:05s-olone sec#2023-06-2208:16s-ol@U09FL65DK sent a DM#2023-06-2208:33braai engineerI'm trying to demo electric-starter-app on a fresh Mac that has Clojure installed via brew install clojure/tools/clojure but am hitting a Cursive exception when opening the project or attempting to start a REPL in IntelliJ:
java.util.concurrent.ExecutionException: com.intellij.execution.ExecutionException: Cannot execute: /usr/local/bin/clojure does not exist
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2096)
at org.jetbrains.concurrency.AsyncPromise.get(AsyncPromise.kt:49)
I can run clj -A:dev -X user/main in terminal just fine, but then I can't do REPLy things.
Hopefully this is a simple linking issue? I also tried brew install clojure.#2023-06-2208:38s-olleave off the -X user/main and it drops you in a repl, then run (main) to start the compiler#2023-06-2208:39s-olthere's probably a way to connect Cursive properly but I don't know that tool#2023-06-2208:41braai engineer@U05AL1ZH8TW clj -A:dev does not seem to start an nREPL, at least not one that I know how to connect to: there is no .nrepl-port file.#2023-06-2208:42braai engineerI realize I can add an nREPL dependency etc., but would really like to be able to use the Local REPL feature in Cursive.#2023-06-2208:43s-olyeah, it's a plain clojure repl. I just did a quick search and it seems that you can configure Cursive to use a 'clojure.main REPL' which is what you want, I think: https://cursive-ide.com/userguide/repl.html#local-repls#2023-06-2208:45braai engineerFIXED: add a symlink like so:
sudo ln -s /opt/homebrew/bin/clj /usr/local/bin/clojure
#2023-06-2208:48s-oljust curious: is there a /opt/homebrew/bin/clojure?#2023-06-2208:49braai engineeryes because I installed Clojure via Homebrew, e.g.
➜ ~ which clj
/opt/homebrew/bin/clj
#2023-06-2208:49s-olI meant clojure (long-form)#2023-06-2208:49braai engineeryes,
➜ ~ which clojure
/opt/homebrew/bin/clojure
#2023-06-2208:50s-olmaybe it expects to find that next to clj, but the symlinking breaks that#2023-06-2209:09braai engineerno, it was looking for /usr/local/bin/clojure which did not exist (and was not symlinked).#2023-06-2209:10braai engineerBoth /opt/homebrew/bin/clj and /opt/homebrew/bin/clojure` are symlinks to files in Cellar paths.#2023-06-2211:07braai engineerI see that hyperfiddle.electric-svg/svg-element, uses dom/with and e/on-unmount to inject SVG nodes. Would this be considered a good reference for dealing with imperative libraries that need access to underlying DOM nodes, e.g. Mapbox GL? For various reasons I don't want to use React go-betweens because it adds a lot Reagent/React complexity and would rather just add map events myself and deal with component lifecycle.#2023-06-2211:21Dustin Getzyou don't need dom/with, you just need dom/node#2023-06-2211:22braai engineerThanks, Dustin. How do I do cleanup on unmount?#2023-06-2211:23braai engineerI see this: https://github.com/hyperfiddle/electric-examples-app/blob/1275f316e68520765446d2c294d4b5ea14cfc4e1/src/wip/js_interop.cljc#L19
Not obvious what the API is.#2023-06-2211:23braai engineere/on-unmount?#2023-06-2211:24Dustin GetzClean up what exactly?#2023-06-2211:24Dustin Getzon-unmount means what it means in reactjs#2023-06-2211:25Dustin GetzYour Mapbox integration is likely not going to be coded in electric, it will manage all the callbacks and events with a missionary adapter#2023-06-2211:25Dustin Getzso your callback allocation/disposal will be managed by m/observe#2023-06-2211:25braai engineerthx, any examples around? BlinkerComponent example is simple#2023-06-2211:25Dustin Getzsearch m/observe#2023-06-2211:26braai engineerWhere should I search? In electric-examples-app I find only this: https://github.com/hyperfiddle/electric-examples-app/blob/1275f316e68520765446d2c294d4b5ea14cfc4e1/src/user/demo_10k_dom.cljc#L14#2023-06-2211:26Dustin Getzin the slack#2023-06-2221:31VincentI'm a little confused on how to achieve POST via cljs-ajax from Client to Server.
(ui/button
(e/fn []
(do (.log js/console !w)
(POST "/create-account"
{:params {:w !w}
:format :text})))
(dom/text "Hit me"))
using cljs-ajax to accomplish the POST but on the server I see
:body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x21eee3a5 HttpInputOverHTTP@21eee3a5[c=0,q=0,[0]=null,s=STREAM]],
when reading ring-req#2023-06-2300:35Dustin Getzi’d guess you’re missing a middleware #2023-06-2300:39Vincentyeah i think so, tried adding some json reading middleware, still looking into it#2023-06-2300:39Vincenti appreciate all the help i really do 😅#2023-06-2301:15Reily SiegelIs it possible to use electric without a backend server, only using the missionary-based reactive dom functions? I'm trying to create an SPA that connects directly to websockets that I don't control. Electric would be a great fit for this, since data is coming in incrementally.#2023-06-2301:19Dustin Getzyes this works natively, just don’t ever call e/server#2023-06-2301:21Dustin Getzyou’ll need to write some missionary to “integrate” whatever events are coming over the websocket into an electric signal #2023-06-2301:23Dustin Getzwe are working on a new primitive e/for-by-streaming that would be suitable for rendering an incremental table derived from a stream of messages, but it’s not going to land until late July #2023-06-2301:25Reily SiegelMy plans for incremental rendering with the current e/for-by was (something like) (m/latest identity (m/reductions conj (sorted-set) my-flow))#2023-06-2301:37Reily SiegelI'll look into seeing if its possible to hack e/boot to not create a server part, and not try to connect to it. (Ideally, there wont be a server to connect to.)#2023-06-2301:37Reily Siegelor are you saying that e/boot wont try and connect to a server ws if e/server is never used?#2023-06-2302:30Reily SiegelThis seems to work as a really hacky replacement for e/boot that disregards any server and works without a backend to connect to, but I wont pretend to understand the macros enough to even really guess /why/ it works
(defmacro boot [& body]
(let [[client] (c/analyze
(assoc &env ::c/peers-config {::c/local :cljs ::c/remote :cljs})
`(e/with-zero-config-entrypoint ~@body))]
`(~(r/emit (gensym) client)
(m/rdv)
identity)))#2023-06-2305:48Geoffrey GaillardYou might want to look at e/run and e/local too.#2023-06-2310:16Dustin Getzsorry, yeah as Geoffrey said, i think the test entrypoints should do what you want #2023-06-2301:45Vincentrather than send something via ajax, is it possible to place something onto a clj channel from a cljs channel? o_O#2023-06-2301:45Vincentajax feeling redundant and not working out of the boxs 😅
got me thinking, maybe leverage the socket, and maybe can put on channel?#2023-06-2301:50Reily SiegelIf you are communicating with your own server, you should be able to just call whatever your http handler on the server is doing, directly from the client, inside a (e/server ,,,).
(d/defn MyComponent []
(e/client (e/server (async/put! a-channel a-val))))
#2023-06-2301:52Vincenthaha hell yeah! that's what i'm talkin' about. okay i don't know much about channels at the moment but i reckon they ought be the thingy.#2023-06-2301:53Vincentasync/put! 😄 cool#2023-06-2301:53Reily SiegelIf by channel you mean a core.async channel, I would look into wrapping it into missionary tasks/flows (or just using missionary tasks/flows instead), since that will be much more directly compatible with electric.
See https://github.com/leonoel/missionary/wiki/Basic-Walkthrough:-Tasks-&-Flows and https://github.com/leonoel/missionary/wiki/Task-interop#coreasync-channels#2023-06-2301:53Vincentoh how exciting, i don't need ajax for talking to the server, duh! lol#2023-06-2301:54Vincenteven sending a collection of data tho^ which is this new case in my app.#2023-06-2301:54Vincentmissionary tasks/flows ponder ponder ponder#2023-06-2301:54Reily SiegelYeah, any clojure value should work.#2023-06-2301:54Reily SiegelAs long as it can get serialized on a websocket.#2023-06-2301:55Vincentso cool#2023-06-2301:55Vincentthanks! i will read up.#2023-06-2310:21Dustin Getzif you are just doing crud stuff you can write to the database in an e/server directly without any async stuff (unless the database client uses core async in which case we recommend you adapt it to missionary, we have adapters for that in the electric repo somewhere)#2023-06-2310:22Dustin Getzwrap blocking APIs with e/offload to move the call to a thread#2023-06-2310:23Dustin Getzthe todos demo has a backend database#2023-06-2302:30Reily SiegelThis seems to work as a really hacky replacement for e/boot that disregards any server and works without a backend to connect to, but I wont pretend to understand the macros enough to even really guess /why/ it works
(defmacro boot [& body]
(let [[client] (c/analyze
(assoc &env ::c/peers-config {::c/local :cljs ::c/remote :cljs})
`(e/with-zero-config-entrypoint ~@body))]
`(~(r/emit (gensym) client)
(m/rdv)
identity)))#2023-06-2315:15Adrian SmithAny speculation on what optimistic updates might look like from a code pov in Electric in the future?#2023-06-2317:00Dustin GetzYou will need to embrace our UI primitives (e.g. use ui/input not dom/input). The upcoming fifth UI design ui5 is callback-free, changes are represented as signals (e.g. an input control is an e/fn that returns a signal of the latest value of the input). There will be a few higher level Form/Field/List components that process/merge these signals and also coordinate the merging of client and server information. At the application entrypoint (at the "top"), the Page will return a signal of requested (pending) edits (as a value) that the entrypoint should transact into whatever database. The view itself should remain fully composable - something like (Form. (div (Field. props (Checkbox.))) (div (Field. props (Input.))) . I think it can be achieved, though the design is pushing Electric very hard and Electric will need some improvements to make it work#2023-06-2317:03Dustin GetzIf the design succeeds — and i don't see any show-stopper risks after working on this for a few weeks — really you shouldn't care or notice any machinery related to optimistic state, it will "just work"#2023-06-2317:03Dustin GetzOne specific tradeoff I'm fairly committed to is to make no attempt to run datalog/database stuff on the client. Any pending edits are considered view-local, not global. That is why there will be a List controller that can merge client state into a collection that came from the server in a very simple and local way – i.e., no database integrations to make pending records show up in queries globally across the client. (Is this what Fulcro does?)#2023-06-2317:09Dustin Getz(That said, you can imagine a future "Electric Datascript" with a network-transparent datalog engine and network-transparent indexes, which would put the whole problem of optimistic queries to rest permanently)#2023-06-2317:11Dustin GetzAlso the UI5 controls know if they are "in sync" or not, so the application can render "sync dots" next to each control if desired (yellow->pending, red->rejected etc)#2023-06-2317:21Dustin GetzIn theory, UI5 control state should survive network unplugs and resync when plugged back in, giving offline-mode capability. Note this is not quite offline-first, we merely wish to guarantee the safety of user state#2023-06-2513:34Dustin Getz@UCJCPTW8J was this what you are looking for? Some form of feedback would be appreciated so I can calibrate#2023-06-2608:12Adrian SmithYeah no problem
This is my use case:
https://clojurians.slack.com/archives/C7Q9GSHFV/p1683796408377749?thread_ts=1683793979.145529&cid=C7Q9GSHFV
I'm trying to recreate a bit of software called MindNode which isn't particularly novel besides the fact that all the interactions are high quality#2023-06-2610:30Adrian SmithI suppose the bit I'm interested in, is how readable or translatable will the optimistic update functionality of the new UI components be for what I'm trying to do
Maybe I can try and use one of them directly, or try and reimplement them for my use case not sure yet#2023-06-2611:01xificurCdo you have business-level requirements listed you can share? E.g. a) locally optimistic, b) synced to server at least every 30ms, c) ... Or a complete user story. Real world use cases like this can help us sharpen the design as we work through the problem space#2023-06-2613:01Adrian SmithWell it's just for fun at the moment
but this is the context of the original app:
https://www.youtube.com/watch?v=urM77gmxnK8
This is an application that is local only at the moment, so the value add of doing it in electric is google docs style collaboration
One of the first things I wanted to do was moving a "node" so you left click on a node, drag it and it mouse up to place it
In the original application, the node moves with your mouse in real time as you drag it, so I wanted to implement that
My priority is that the application moves the "node" as soon as the mouse event registers, then sends that position across the network when it is able to
I care a lot about the current location of the node, but I don't mind if in-between mouse events are lost too much
When I come to implement undo I'm not going to care about per pixel location but instead restore to checkpoint locations that will be marked with mouseup/down
I don't know if that helps, it's kind of similar but not quite the same requirements as a multiplayer drawing app#2023-06-2613:09Dustin GetzI think the form-centric approach shares the exact same underlying utilities needed for syncing mouse signals#2023-06-2613:10Dustin GetzSyncing a mouse signal is much simpler actually, the approach used in UI4/input i think can be slightly modified to work for mouse signals#2023-06-2613:12Dustin GetzUI4 control implementation is complicated though, we aren't happy with the low level API at all, it was an early POC, we went with it in UI4 because it works and fixed problems#2023-06-2613:15Dustin GetzTo answer the q directly: "how readable or translatable will the optimistic update functionality of the new UI components be for what I'm trying to do":
the userland api should be perfectly readable#2023-06-2321:42Vincentif i want to add an external cljs library to the front-end (like Stripe.js) is there a way i can add the javascript file more natively than loading it via html?#2023-06-2321:44Dustin Getzshadow-cljs understands node modules iirc, im a bit fuzzy on the details but you should be able to find info on clojure verse and searching other channels on this slack#2023-06-2321:55VincentchatGPT4 say there is a mystical deps.cljs one can invent to add external js libs. am wondering if it same tactic#2023-06-2321:56Geoffrey Gaillardhttps://shadow-cljs.github.io/docs/UsersGuide.html#js-deps#2023-06-2322:07Vincentnice i hear my brain sizzling already#2023-06-2322:11VincentAlways load Stripe.js directly from to remain PCI compliant. Do not include the script in a bundle or host a copy of it yourself.
:melting_face:#2023-06-2323:26Dustin Getzit’s 100% ok to load by script tag, you might need to give clojurescript advanced mode a few hints if you care about that#2023-06-2400:34VincentHey I finally got it to render 😄
I might have to make some instructional videos on un-learning conventional client/server comms#2023-06-2406:35Geoffrey GaillardYou can use the Stripe.js npm module, it will lazy load the JS script from their CDN. It will save you a split config and might help with advanced mode.#2023-06-2322:11VincentAlways load Stripe.js directly from to remain PCI compliant. Do not include the script in a bundle or host a copy of it yourself.
:melting_face:#2023-06-2413:27bahulneelHas anyone had experience running electric as a service worker, I have an existing UI built in Nuxt that I want to integrate with an electric+xtdb backend? Is this event the correct approach, I’d appreciate any comments/suggestions.#2023-06-2413:28Dustin Getzwhat is a service worker#2023-06-2413:29bahulneelIt allows you to run JS (with limitations i.e. no DOM) in a separate thread in the browser.#2023-06-2413:30bahulneelYou communicate with the page via message passing#2023-06-2413:31bahulneelsorry, I mean web workers, service workers are similar but network traffic specific#2023-06-2414:27Dustin Getzby "run electric as a web worker" you mean you want to run electric entirely in a thread, so as to be isolated from Nuxt, keeping the Nuxt UI and yet talking to an Electric backend, so as to load data from electric into a clientside datastore for your Nuxt components? What exactly do you want to do?#2023-06-2414:29Dustin GetzI would ask you to show me some application code sketches of what your code looks like today (highlighting the problem you want to solve) and also what you would like your code to look like#2023-06-2414:46bahulneelAfter reading your documentation I'm not exactly sure what I want to do. I'll go and think about it and come back with something concrete.#2023-06-2415:01Dustin Getzfwiw we are just now starting a Fulcro integration, which is of the form of embedding an Electric component inside the Fulcro/React component tree, which can accept props from Fulcro and interact with any services in scope (like routing). In this worldview the electric subtree performs its own IO where desired while also able to receive data from Fulcro#2023-06-2415:03bahulneelInteresting, I think I need something similar. Is there a good place to look at integration patterns/options#2023-06-2415:06Dustin Getzthis slack channel indexes everything interesting, i dont recall anyone having worked this out deeply yet#2023-06-2415:12bahulneelOk thanks, I think I need to grok electric a bit more, the answer may be out there already. I'll probably implement the basis of what I want to achieve, starting with the xtdb starter app, without the integration and then, at least, I'll know what I need to solve.#2023-06-2419:54Sagar VrajalalMaybe a very silly question but how do I go about changing the default page title? I copied the server code from electric-starter-app
<head>
...
<title>Hyperfiddle</title>
...
</head>
#2023-06-2419:54Dustin Getzedit index.html#2023-06-2419:55Dustin Getzif you want to set it dynamically, you'll need a side effect in your router code#2023-06-2420:03Sagar VrajalalI actually wasn't using index.html . That fixed it. I would like to set it dynamically though, could you explain what you mean by a side effect in your router code? I don't quite understand how it gets generated when it's missing.#2023-06-2420:04Dustin Getzindex.html is not generated, it's a static resource in the starter app, Electric is solely a websocket abstraction#2023-06-2420:05Dustin GetzIt's up to you how to generate index.html, the only requirement is that you integrate our websocket middleware#2023-06-2420:06Dustin Getzhttps://biffweb.com/p/how-to-use-electric/ may help if you'd like a more framework-y approach#2023-06-2420:07Dustin Getz(set! (,-innerText (js/document.querySelector "title")) "hello world") (pseudocode) is what i meant by side effect#2023-06-2420:13Sagar VrajalalThanks. I was already doing JS interop to include stylesheets! I guess this was more of a CLJS/JS DOM manipulation question more than anything else.#2023-06-2420:14Dustin Getzoh, you have your own backend already it seems#2023-06-2420:15Dustin GetzSorry, i dont quite understand your setup but glad you're sorted#2023-06-2420:18Sagar VrajalalBasically instead of using index.html I did something of the sort :
(defn add-tag
[{:keys [tag id props]}]
(when-not (.getElementById js/document id)
(let [element (.createElement js/document tag)]
(doseq [[attr-key attr-val] (merge props {"id" id})]
(.setAttribute element attr-key attr-val))
(.appendChild (.-head js/document) element))))#2023-06-2420:19Sagar VrajalalThe rest is a modified version of xtdb-starter-app#2023-06-2420:19Dustin Getzwhat serves the script tag?#2023-06-2420:20Sagar Vrajalal(def electric-main
(hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
(binding [hyperfiddle.electric-dom2/node js/document.body]
(set! (.-textContent (js/document.querySelector "title")) "Hello World")
(mapv add-tag meta-tags)
(Main.))))#2023-06-2420:21Sagar VrajalalI'm quite new to frontend, JS and CLJS in general, so I'm just experimenting#2023-06-2420:22Sagar Vrajalalsorry, not sure what you mean?#2023-06-2420:22Dustin Getzsomeone has to serve the frontend js entrypoint and inject it into the browser so it runs#2023-06-2420:22Dustin Getz<script src="app.js">#2023-06-2420:27Sagar VrajalalAs I said I'm just using the server code from electric-starter-app so I'm not sure how that's working.
https://github.com/hyperfiddle/electric-starter-app/blob/main/src/electric_server_java11_jetty10.clj
I can see the script tag in the body
<script type="text/javascript" src="/js/main.js"></script>
#2023-06-2420:29Dustin Getzah ok, that starter server does indeed serve index.html from the resources#2023-06-2420:29Sagar VrajalalI think shadow-cljs is doing it?
((requiring-resolve
'shadow.cljs.devtools.server/start!))
((requiring-resolve
'shadow.cljs.devtools.api/watch) :dev)
;; Start electric compiler and server
(ring/run-jetty
(http-middleware "public" "public/js/manifest.edn")
opts)
#2023-06-2420:30Sagar VrajalalBecause I'm not using index.html#2023-06-2514:54s-ol@U01B1CZQ9PF: the electric starter app contains an index.html in resources/public: https://github.com/hyperfiddle/electric-starter-app/blob/main/resources/public/index.html
There always has to be a HTML page, it's not possible to have a website without it. It could be generated "on the fly" instead of being loaded from disk, but the browser needs HTML to load a website, since it is only the HTML that can tell the browser what JS code to load.#2023-06-2515:09Sagar VrajalalThank you, thats helpful.#2023-06-2503:09AmosHi, I wonder whether electric can support desktop application development.#2023-06-2513:32Dustin Getzit theoretically can (with minor changes to the entrypoint) but it is not a priority#2023-06-2513:59AmosWe are trying to build an app that can support web and also desktop. I wonder what do you mean about the minor changes to the entrypoint.#2023-06-2514:22Dustin GetzWhich UI library would you want to use? the equivalent of electric-dom would need to be built for the native rendering#2023-06-2514:22Dustin Getzelectric-dom is 300 LOC, it's not hard#2023-06-2604:46AmosWe are plan to use electron js to build desktop app. Also, we will like to use electric to communicate between desktop app and the remote server. However, we found difficulty to do this. (Since in the electric, the frontend code and the backend code is written together, If we want to treat the desktop as the client to communicate with remote server, it will be hard).#2023-06-2606:36KeinAmos an I wonder what the possible electric configuration might be needed to pack the fullstack project into electron.#2023-06-2610:54xificurCthe server<->client communication happens over a websocket. If you can create a websocket connection from electron the configuration should be similar#2023-06-2611:46Dustin Getz@U0577BSF2CC the electric jetty server can be hosted by the remote server, e.g. add electric to your existing server project#2023-06-2611:47Dustin GetzWe do not currently test with nodejs clients however, I imagine minor changes will be necessary. The electric language test suite passes on nodejs but we do not run any real apps that way#2023-06-2611:49Dustin Getzto be very transparent: Nodejs support is not a priority for us because we don't make any money from that; it will become a priority when someone hires us to do it#2023-06-2614:20AmosThanks for your information. We will try to look into that.#2023-06-2614:59AmosI understand that I host the electric jetty server on the remote server. However, the client code will not be on the server (will be on the desktop). I think this is one of the difficulty we have faced.#2023-06-2615:00Dustin GetzIIUC - the desktop client and the server can be in the same project, allowing the electric compiler to see everything#2023-06-2615:01Dustin Getzwhether or not the client is on a desktop or in a browser should not make a difference wrt network sync#2023-06-2615:01Dustin Getzthe key is that the electric compiler can see both sets of namespaces#2023-06-2615:39AmosI wonder when you say ‘desktop client and the server can be in the same project’, is the project compiled on the server or desktop?
If it is on the desktop, this means that the server and the client will both be inside the desktop app. Do I understand it correctly?#2023-06-2616:30KeinWhen we try to investigate packing the clojure/script app into Desktop via frameworks like Electron and Tauri, it should be correct to view the project as a separate client server. Electron/Tauri handles the client-side code, while we deploy a Clojure process on the server. However, understanding it is not trivial to us as Electric made both sides of code seemingly coupled as a whole.
Do I hypothesize correctly that:
[1] When we start the server, the server do not need to know the DAG, as defined in e/defn scope (I hypothesize this because I didn’t find that running electric server has any dependencies of client-side code/DAG such as the app.todo-list )
[2] When the client connects to the server, the client sends the DAG to the server , and [1] e/server functions get defined on the server [2] the web socket is established [3] the DAG and the protocol is established. (This seems nearly impossible but it might be approachable as code is data.)
What would be the more correct dynamics, or some basic facts around the hypothesis?#2023-06-2618:09Dustin Getz@U0577BSF2CC webapps like electric-starter-app are compiled twice – there is a clojure build and a clojurescript build. In the project you describe, i would expect a clojure build targeting jetty and a clojurescript build targeting electron. The point is at compile time the two codebases are unified into one so that the compiler can see everything and coordinate the network boundary.#2023-06-2618:11Dustin Getz@U03RQQ2L8LE your questions [1] and [2] are about implementation details and subject to change (in fact the whole Electric architecture is subject to change, we are gradually moving more things to runtime like the JVM)#2023-06-2618:19Dustin GetzSpeaking to today's implementation, the DAG is a value (imagine a buffer of bytecode) and that value is produced as part of the ClojureScript build, the server actually does not do any Electric compiling today, the client at runtime passes a DAG value (electric bytecode) up to the server in the first websocket message. (Yes that means in prod you should validate the hash of the DAG value.) This is an implementation detail and we plan to change it this summer.
"e/server functions" is not a thing, all Electric code has been compiled down to electric bytecode at this point. The Electric bytecode contains opcodes for Clojure interop, i.e. to resolve a Clojure var or call a Clojure function like println . Calling an electric function (an e/fn on the server) is not a thing because all Electric code has been compiled down to bytecode, and then the bytecode furthermore is either interpreted or compiled down to Missionary flows. So there are no "electric functions" in the end, because Clojure is not reactive, we have to compile it into reactive target code, in this case missionary flows are the compile target.
the Electric compiler pipeline is: DAG -> bytecode -> missionary flows in Clojure/Script -> [Clojure toolchain ... ]. There is also a runtime (e.g. an Electric virtual machine for the Electric bytecode, note both the "VM" and bytecode are distributed)#2023-06-2503:18VincentAfter much wrestling with a delicate situation, I got some stripe support working. To be PCI-compliant you must create a paymentIntent from the server (e/server (POST "stripe com paymentintents endpoint" {amount currency})) and then pass that to the client, and package the clientside inputs with that secret when you send it off to stripe
with Electric, this looks kinda like
#?(:cljs (def !stripe-result (atom "")))
(e/def (sr (e/watch !stripe-result)))
Then,
... in some component ...
(e/client
(ui/button
(e/fn []
(e/server
(POST to stripe, save result to atom)
(e/client
(do something with sr)))))#2023-06-2513:16Dustin GetzThere are easier ways to integrate stripe, for example you can link to a payment form hosted by them and manage the whole thing out of band until you have dozens of checkouts per month#2023-06-2513:21Dustin GetzAlso, the ui4/button and various callback patterns that use dom events can cause subtle accidental loops, you need to be really careful not to smash that API by accident#2023-06-2513:24Dustin Getz(UI5 will hopefully address any loop issues that we haven't already addressed, and we've addressed most of them, but Electric is 100% experimental/alpha and perhaps a payments integration can be delayed for a while until all the UI idioms stabilize)#2023-06-2520:33VincentI only hit the endpoint I really care about once 😅 I'm looking at the network tab and realizing it does hit shared.js more than is required, i may go the route of creating a separate application page for it, but i really wanted to push and see if it could be all electric, and it can, which is very cool, some "synchronous" behaviour with button clicks and polling the page to check if stripe.js is loaded, kinda janky kinda hacky, but working is much nicer than not yet functioning 😄#2023-06-2503:19VincentI ran into several snags trying to do java interop and ended up resorting to http calls to stripe (but from the e/server )
so yeah, stuff like this is actually really quite fun to program (when you can build incrementally from some working program)#2023-06-2515:50sansaripHello blob-wave In regards to routing I see a https://clojureverse.org/t/electric-clojure-a-signals-dsl-for-fullstack-web-ui/9788/28 for electric! Sorry if it’s obvious, but where may I locate it?#2023-06-2611:56Dustin Getznot obvious, found it in a scratch, here is everything we have for retit:#2023-06-2611:56Dustin Getz#2023-06-2611:58Dustin GetzThis came from @U050CJFRU , Dennis are you still using retit and if so can you paste me or DM a paste of your integration that I can cleanup?#2023-06-2612:23nakkayaI am also using retit like the following,
#?(:cljs
(def router
(r/router
[["/patients" ::patients]
["/add-patient" ::add-patient]
["/view-patient/{uuid}" ::view-patient]
["/view-radiology/{image-uuid}/{patient-uuid}" ::view-radiology]])))
#_:clj-kondo/ignore
(e/defn Page []
(let [match (r/match-by-path router browser/path)]
(if-let [view (get-in match [:data :name])]
(let [params (:path-params match)]
(condp = view
::patients (Patients.)
::add-patient (AddPatient.)
::view-patient (ViewPatient. params)
::view-radiology (ViewRadiology. params)
(Patients.)))
(Patients.))))
#2023-06-2614:05denikMost up to date here: https://github.com/lumberdev/tesserae/blob/cea33f19b46892abb78feb99d51af2dd54849435/src/tesserae/ui/app.cljs#L15#2023-06-2517:07s-olWhats the equivalent of the "React key prop" for resetting local state in reactive functions outside of loops?
E.g. when using a router and navigating between two routes /obj/123 and /obj/345 that call the same view component (ObjView 123) / (ObjView 345) I want to un & re-mount that call when the id changes, thereby re-initializing e.g. any atoms defined within rather than propagating the change further through the dag#2023-06-2518:24Dustin GetzI suspect there may be a better way than that, can I see any sample code you have that makes you reach for this pattern?#2023-06-2518:28s-olsee dm#2023-06-2519:13Dustin GetzTo close the loop in public - after reviewing s-ol's code, I recommended a little hack like ((fn [_] (atom 1)) id) to reset the state rather than rebooting the view, we really do not want people "rebooting" Electric functions the way we reboot React components#2023-06-2519:16Dustin Getz(We could provide a macro do achieve it, by like "blinking" the component to nil or something, but i would want to see a lot more use cases to justify it, and even here the "hack" seems sensible in the context of what the code was doing, which was resetting an infinite scroll state)#2023-06-2611:04grounded_sageInfinite gratitude. Having the GitHub actions and Dockerfile is a godsend for getting up and going quickly. #2023-06-2611:06grounded_sageIs there any open repositories laying around that implements durable storage on Fly or other platforms?#2023-06-2611:37Dustin Getzwe haven't done durable storage on fly yet, it's going to be harder due to the geographical distribution that fly is meant to enable#2023-06-2611:37Dustin Getzi recommend getting started with a centralized instance, i.e. digital ocean or something#2023-06-2611:39Dustin GetzThe reason our demos use fly is because the demos are realtime and need to be fast for users around the world, but most apps don't need that and/or can backlog it for a while until you can afford the devops work#2023-06-2611:41Dustin GetzYou can also configure fly to limit to a single instance in a specific datacenter, and then connect to the closest AWS region over public internet. But again the devops work will be more complicated than simply using a classical host until you outgrow it.#2023-06-2611:45grounded_sageCool. Yea I was thinking it might be possible to use Datomic and connect it to their Postgres. I’ll have a poke at it and figure out my angle for v1 of what I am building. #2023-06-2611:53grounded_sageI’ve done very little devops. But Fly looks easier to me than wrangling AWS as a solo dev. I was thinking one instance is set up with their Postgres and Datomic Transactor. Then the rest are peers. #2023-06-2611:55grounded_sageHmm actually not sure that would work because the transactor would still be swapped out with new updates. So Postgres would have to be on its own instance. I see.#2023-06-2611:59Dustin Getzyeah i looked into Fly managed Postgres and it didn't look ready last i checked#2023-06-2611:59Dustin Getzglobally distributed ACID is still hard, you don't need to solve this#2023-06-2612:05grounded_sageI guess what I am trying to say is I have energy to contribute to a streamlined workflow for a durable quick start repo. If it is aligned. #2023-06-2612:08Dustin GetzSure, we would accept that contribution#2023-06-2711:25braai engineerInstalled Datomic dev-local and installed the samples (I can enumerate the sample sets), but when I run electric-datomic-browser I get this error:
👉 App server available at
#error {
:cause No implementation of method: :recent-db of protocol: #'datomic.client.impl.shared.protocols/Connection found for class: clojure.lang.Var$Unbound
:via
[{:type java.lang.IllegalArgumentException
:message No implementation of method: :recent-db of protocol: #'datomic.client.impl.shared.protocols/Connection found for class: clojure.lang.Var$Unbound
:at [clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 584]}]
:trace
[[clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 584]
[clojure.core$_cache_protocol_fn invoke core_deftype.clj 576]
...#2023-06-2711:25Dustin Getzread user.clj#2023-06-2711:27braai engineerI have - don't see what I should do? datomic-client and datomic-conn in user.clj seem fine:
(d/db datomic-conn)
=>
#datomic.core.db.Db{:id "mbrainz-subset",
:basisT 711,
:indexBasisT 0,
:index-root-id nil,
:asOfT nil,
:sinceT nil,
:raw nil}
#2023-06-2711:27braai engineerapp.queries not in starter though:
(tests "healthcheck"
(->> (m/? (app.queries/query
'[:find (pull ?tx [:db/id :db/txInstant])
:where [?tx :db/txInstant]]
db))
(take 1))
:= [[#:db{:id 13194139533312, :txInstant #inst"1970-01-01T00:00:00.000-00:00"}]])#2023-06-2711:35braai engineerAh, it just started working after waiting a while. Something must have been loading in the background.#2023-06-2713:05Dustin GetzI dont recall seeing that before#2023-06-2713:06braai engineerthere was a "Loading...." message somewhere in the Datomic logs.
Probably because it was a fresh datomic install with fresh samples#2023-06-2714:30AmosHi, I’m looking into the electric jetty code. I wonder does the jetty web server only handle the backend (clojure) code? or it also process the frontend code (After compiled by the shadow-cljs)? I have tried to look into the http-middleware function. However, I’m still not sure about this part.#2023-06-2714:50xificurCHere is a rough explanation of how code turns into a runnable web page today
- one starts a build
- shadow-cljs is called to compile js sources
- all code is bundled in an uberjar
- one deploys the jar
- a webserver (jetty) starts
- when you open a web page jetty serves index.html
- index.html requests the js sources through a <script> tag
- the electric program boots from these js sources
- the client connects to the webserver
- the client sends the server part of the electric program to the server
- the server boots the server part of the electric program
Does that answer your question?#2023-06-2714:58Dustin GetzWe need to separate the idea of "clojure/script code" from "electric code"#2023-06-2714:58Dustin Getze/fns compile down to "DAG values" ("electric bytecode") and that bytecode contains opcodes for Clojure interop (var lookups and clojure function calls)#2023-06-2715:00Dustin Getzthe jetty server will evaluate the DAG bytecode and perform any Clojure var lookups and Clojure function calls. the "electric code" is basically erased, it compiled down to bytecode. The bytecode is then either interpreted into Clojure missionary flows (on the server), or compiled into ClojureScript missionary flows (on the client). The server's electric bytecode interpreter does the var lookups and clojure function calls. The client electric bytecode is futher compiled bytecode -> CLJS -> JS because ClojureScript doesn't have vars#2023-06-2715:15Dustin Getz> does the jetty web server only handle the backend (clojure) code? or it also process the frontend code
the jetty server sees the electric DAG bytecode, and IIRC both client and server get the complete bytecode for the electric system today. But the jetty server will not see any ClojureScript code. And the jetty server's only relation to Clojure code is that it interprets the electric opcodes for clojure var lookup and clojure function call#2023-06-2718:44Stephan RenatusHiya! Are there any links for these? https://github.com/hyperfiddle/electric/blob/master/docs/electric-apps-in-the-wild.png#2023-06-2718:45Stephan Renatuscontext is just that I’m curious to see more real-world examples#2023-06-2719:31Dustin Getzhere are a couple more
• https://electric-demo.fly.dev/(user.demo-explorer!DirectoryExplorer)
• https://electric-demo.fly.dev/(user.demo-todomvc!TodoMVC)#2023-06-2719:31Dustin Getzhttps://github.com/hyperfiddle/electric-datomic-browser#2023-06-2719:32Dustin Getzthe best crud app demos are private unfortunately#2023-06-2806:31Stephan Renatusthank you!#2023-06-2909:42Lidor CohenHello! 🙂
I'm trying to get rcf working in my shadow-cljs project:
• I have this (and only this as regarding to rcf) in my shadow-cljs.edn under :dependencies:
[com.hyperfiddle/rcf "20220926-202227"]
• I have this dev file:
(ns dev.core
(:require [spiritt.experiments.obj-tests]
[portal.web :as po]
[hyperfiddle.rcf :refer [tests]]))
; wait to enable tests until after app namespaces are loaded
(hyperfiddle.rcf/enable!)
; subsequent REPL interactions will run tests
; prevent test execution during cljs hot code reload
(defn ^:dev/before-load stop [] (hyperfiddle.rcf/enable! false))
(defn ^:dev/after-load start [] (hyperfiddle.rcf/enable!))
(comment
(po/open)
(add-tap po/submit)
;;
)
• And I have these tests:
(ns spiritt.experiments.obj-tests
(:require ...
[hyperfiddle.rcf :refer [tests]]))
(hyperfiddle.rcf/enable!)
.
.
.
(tests
(get-slot spatial-method :b) := "TEST1 SUCC1"
.
.
.
;;
)#2023-06-2909:47Lidor CohenNow, I'm not sure what to expect, I'm trying evaluating tests and I get nil.
Loading the file doesn't do anything.
(I'm using vscode + calva BTW)#2023-06-2910:03Dustin Getzcheck browser console#2023-06-2910:08Lidor Cohennothing...
it should happen when I eval tests?#2023-06-2910:49xificurCare these cljs files?#2023-06-2910:50xificurCthe cljs hooks you copied turn rcf off while the namespaces are loaded#2023-06-2910:51xificurCto run them manually one needs to connect a cljs REPL and eval the tests blocks#2023-06-2910:52xificurCfor our main repo we run the cljs tests as part of our CI pipeline. https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/ci/run_tests_browser.sh, also check https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/shadow-cljs.edn#2023-06-2911:03Dustin GetzFor CLJS tests to run, rcf/enable! must be true in both clj (macroexpansion time) and cljs (js runtime). Successful runs will print green checkboxes in the browser console; clojure REPLs don't intercept the async print properly#2023-06-2911:17Lidor Cohen🙏 I'll give it a try#2023-06-2911:55Lidor CohenGot it to work 🥳#2023-06-2911:56Lidor CohenWill send here all that's needed for the simplest setup with cljs and shadow#2023-06-2912:00Dustin GetzAppreciate that thank you, I'd like to understand what you were missing before#2023-06-2912:05Lidor Cohenshadow build-hook:
:build-hooks [(dev.core/rcf-shadow-hook)]
hook itself clj file: (dev/core.clj)
(ns dev.core
(:require [hyperfiddle.rcf]))
(def rcf-enable! (delay @(requiring-resolve 'hyperfiddle.rcf/enable!)))
(defn rcf-shadow-hook {:shadow.build/stages #{:compile-prepare :compile-finish}}
[build-state & args]
;; NOTE this won't prevent RCF tests to run during :require-macros phase
(case (:shadow.build/stage build-state)
:compile-prepare (@rcf-enable! false)
:compile-finish (@rcf-enable!))
build-state)
cljs before-load & after-load
(defn ^:dev/after-load start []
(js/console.log "Starting...")
(hyperfiddle.rcf/enable!))
(defn ^:dev/before-load stop []
(js/console.log "Stopping...")
(hyperfiddle.rcf/enable! false))
#2023-06-2912:05Lidor CohenI was missing the clj biuld-hook#2023-06-2912:06Dustin Getzoh crap#2023-06-2912:08Dustin GetzIIRC that is optional, are you saying it is necessary? I believe the purpose of the shadow hook is to prevent JVM tests from running during shadow reloads related to macro namespaces#2023-06-2912:10Lidor CohenI think it's what enables rcf in clj:
:compile-finish (@rcf-enable!)
#2023-06-2912:10Lidor Cohenbut I'm far from sure#2023-06-2912:10Dustin Getzah, that line can be moved to an entry ns like dev.cljc#2023-06-2912:11Lidor CohenBut your message: https://clojurians.slack.com/archives/C7Q9GSHFV/p1688036585861269?thread_ts=1688031756.398229&cid=C7Q9GSHFV
clued me as nothing in my code enabled rcf for clj#2023-06-2912:12Lidor CohenMy project is exclusivly cljs (shadow)
my entry is cljs file I'm not sure how to incorporate a cljc entry point...#2023-06-2912:12Lidor Cohenin shadow it was the build hook for me#2023-06-2912:12Dustin Getzi just added that sentence to the readme fwiw#2023-06-2912:13Lidor Cohen👍 it what helped me 🙂#2023-06-2912:15Dustin GetzI ticked this, thanks#2023-06-2912:15Dustin Getzhttps://github.com/hyperfiddle/rcf/issues/78#2023-06-2913:22braai engineerAnyone have a mySQL Electric example lying around to refresh all viewers anytime anything changes?#2023-06-2913:24Dustin Getzafaik mostly this has been hacks, nobody has done a real integration with a tx-notification-queue of some sort#2023-06-2913:25Dustin Getzthe typical hack is to wrap the transact call and increment a counter out of band to force refresh queries#2023-06-2913:25braai engineerOK so hack is: central server-side counter and any time tx just increment counter. that will work fine for now 🙂#2023-06-2913:25Dustin Getzanother easy approach is to refresh queries when the user navigates like its 2010#2023-06-2916:05braai engineerI can see data/!counter being incremented on tx, but (data/query-geom ...) does not rerun. How do I force it to re-run when data/!counter changes?
(e/defn App []
(e/client
(let [cnt (e/server (e/watch data/!counter))]
(dom/div
(dom/text cnt "hi: " cnt " ok")
(dom/pre
(dom/text
(e/server (pr-str (data/query-geom data/conn)))))))))
;; In data I (swap! !counter inc) on tx.#2023-06-2916:10braai engineerDo I have to pass counter as an argument to my query function? e.g. (data/query-geom hacky-counter conn args ...) ?#2023-06-2916:56Dustin Getzyes, expressions re-evaluate exactly when their arguments change#2023-06-2916:57Dustin Getzyou can wrap it if you want, ((fn [_] (query-geom conn)) cache-bust)#2023-06-2915:39denik#2023-06-2915:39denikstuck trying to run code based on dom events. the goal is to run code inside an electric view every time the window is focused. eu/window-focus is a flow observing and yielding the focus event#2023-06-2915:42denikbasically looking for a (whenever <x (do y)) I know that one way is to pass x into the function inside the when expression. however, since the value is a js object and there’s a server boundary it does not serialize. I could write some more code to return a serializable value instead (e.g. (rand)) but wondering if there’s a more elegant/idiomatic approach#2023-06-2916:20jjttjjNot sure this is useful, the code for checking for hover/focus on a node:
https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/src/hyperfiddle/electric_dom2.cljc#L235-L258#2023-06-2916:47Dustin Getzthe undocumented APIs https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/src/hyperfiddle/electric.cljc#L447-L537 (that UI4 uses) is the current way to do this, the problem is that they are overcomplicated, we only released them because they fixed bugs in UI4#2023-06-2916:52Dustin GetzThey handle edge cases like, what happens if the window focus events are in rapid succession, faster than latency, and in the "event callback" there is an e/server.
• Do you want to cancel and discard the previous callback and start over with the new value? use e/for-event-pending-switch
• Do you want to run N callbacks concurrently? use e/for-event-pending
• Do you want to backpressure the user, disabling the control to block them from submitting events so fast? e/do-event-pending#2023-06-2916:52Dustin GetzYou can try to read https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_ui4.cljc - search for "-event"#2023-06-2916:54Dustin GetzThis over-complicated code is why UI5 uses a completely different pattern, no more callbacks at all, but we are not close to shipping it#2023-06-2915:39denikhttps://github.com/lumberdev/tesserae/blob/master/src/tesserae/ui/electric_util.cljc#L60#2023-06-3019:54jjttjjMessing with the electric_codemirror stuff in contrib a bit.
I need to register a callback for a codemirror keybinding command callback that can run e/server code.
Is the way to approach this wrapping the whole CodeMirror creation code in an m/observe, and handling all key bindings there? Are there any good examples to look at for this?
(let [txt (e/server (slurp "temp/basic.clj"))]
(new cm/CodeMirror ;;from electric_codemirror.cljc but with ability to set :extensions
{:parent dom/node
:extensions
[(.of view/keymap
(clj->js
[{:key "Ctrl-c Ctrl-c"
:run (fn [editor-view]
(println "here")
;; how to call e/server here?
)}]))]
}
identity identity
txt))#2023-06-3020:36Dustin Getzthe idea of using e/server in a “callback” exposes you to these problems : https://clojurians.slack.com/archives/C7Q9GSHFV/p1688053176764759#2023-06-3020:38Dustin Getzthe quickest way to make progress is to convert the events into signals as soon as possible in the adapter (without any e/server support or electric callbacks) and return signals “up the stack” to electric code#2023-07-0107:38nivekuilwhat does this (n) do here? It's not returned from the function and the side effect is just reading the current time? https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/src/hyperfiddle/electric.cljc#L156#2023-07-0111:58Dustin Getzthis is a low level implementation of missionary’s flow protocol, n t is notify and terminate; notify means “a new value is ready” and it is called immediately because this clock always has a value ready which allows it to be consumed as a continuous flow#2023-07-0112:03Dustin Getzthe flow contract is, the producer notifies, and the consumer on notification uses deref to pull a value through. discrete consumers will pull the value through immediately in response to the notify (thereby eager) and continuous consumers will wait to be first sampled themselves before pulling the value through (thereby lazy)#2023-07-0112:06Dustin Getzcontinuous producers wont even compute the value until it is pulled through (lazy sample) with deref; discrete producers will compute the value before notifying (eager push)#2023-07-0117:21nivekuilthanks, I think this tangible, low-level example of a flow and your explanation demystified missionary better than any other example i've seen#2023-07-0111:37Ronny LiDumb question: can I use Electric if I'd like to have a website and mobile app both talk to the same backend?#2023-07-0111:51Dustin Getzsure, you’ll run an electric websocket middleware and a regular http setup side by side, both can call the same functions#2023-07-0111:53Dustin Getzto be clear - electric is targeted at rich dynamic web apps, no native mobile support anytime soon and also no SSR anytime soon#2023-07-0111:53Dustin Getzbut yes you can run electric side by side with any other technologies even in the same server#2023-07-0112:00Ronny LiThank you!#2023-07-0113:56grounded_sageHow straightforward would it be to package up the electric front end code so it could be used as a widget on arbitrary sites?#2023-07-0115:19Dustin Getzcan you give me an example, do you mean like as a NPM dependency for react apps?#2023-07-0115:36grounded_sageKind of. Like a script that someone can use on their site.
I’ve just built it in UIx for now. But curious if this would be possible. #2023-07-0121:01Dustin Getzi think it should mostly just work, the frontend is just clojurescript so compile down to a js file and script tag it in somewhere. the boot sequence might need slight modification to connect to a custom websocket url#2023-07-0314:13grounded_sageI just changed the *ws-server-url* for now. Will be digging into the source a bit more as I go through this. With this I got it working with just a script tag. Things I have noticed though is that my application UI only loads when the websocket is connected and it disappears when that connection is dropped. I'd love to have a rich UI that still renders regardless of the websocket connection. Actually would prefer that the UI renders and the websocket connection is established when my application is activated. It's basically a floating chat widget for reference.#2023-07-0314:21grounded_sageNot a show stopper though#2023-07-0314:32Dustin GetzNice! The application reset issue is unblocked by the Electric v3 workstream which will land soon this summer#2023-07-0400:07grounded_sageWhere can I track that development?#2023-07-0400:17Dustin Getzall releases are announced here, and major releases announced on clojure social media channels#2023-07-0400:23grounded_sageCool. But the rest of development is happening privately right? I was just unsure if I missed something. I don’t mind about the process just curious.#2023-07-0400:39Dustin Getzwe sometimes use github issues for major things, you can open one there to track this if you like#2023-07-0208:02braai engineerIs there an up-to-date Datomic ~Cloud~ (On-Prem because cloud lacks d/listen) Starter somewhere?
contrib.datomic-m and contrib.datomic-contrib are missing from the latest Electric releases (I assume they are being moved out).#2023-07-0209:30braai engineerFound this. Latest? https://clojurians.slack.com/archives/CL85MBPEF/p1685970532152309?thread_ts=1684166929.251079&cid=CL85MBPEF#2023-07-0212:08Dustin Getzthis was an accident#2023-07-0212:14Dustin Getzwe intend to migrate all of our demos to datomic pro and just stop supporting cloud for now #2023-07-0212:15Dustin Getzthis will mean the datomic-contrib namespaces should get a lot simpler #2023-07-0208:23braai engineerNow that it’s free, has anyone managed to deploy a Datomic Pro transactor to http://Fly.io?#2023-07-0212:10Dustin Getzcan you use another host? fly is unnecessarily tricky for most apps, we use fly for our demos because they are accessed worldwide#2023-07-0219:50braai engineerI use Fly because it’s the new Heroku - convenient. Aside from multi-region DBs, why is it trickier than other hosting?
(I wouldn’t bet against Fly long-term – I want something I can scale up memory/CPU from the CLI.)#2023-07-0219:50Dustin Getzmulti-region persistence is why#2023-07-0219:51braai engineeryeah I spent wayyyy too long trying to get litestream working.
In my case, I just want something I can deploy to now, have a central DB in one region with some caches in other regions, and figure out multi-region persistence later.#2023-07-0219:51Dustin Getzfly is not that#2023-07-0219:51braai engineerwhat would you recommend?#2023-07-0219:53Dustin Getzin the past i've used digital ocean, aws, gcp; the up-and-coming would be railway, render etc#2023-07-0219:53Dustin GetzHere is a list of VPS hosts:
• Digital Ocean
• Vultr
• https://www.kamatera.com/express/compute/
• https://aws.amazon.com/lightsail/
• https://www.linode.com/
• https://www.cloudsigma.com/
• https://www.cloudways.com/en/
• https://www.togglebox.com/
• https://www.atlantic.net/
• https://www.scaleway.com/en/
• https://www.interserver.net/vps/?id=517869
#2023-07-0208:38braai engineer@dustingetz is there a more idiomatic way to force Electric to update when DB changes (assuming I control all txes and can trigger an update) than using the !db-time hack that leaks into every query fn? Maybe using add-watch on a central atom + m/observe?
https://github.com/hyperfiddle/electric/blob/1f683adc6552be0f3f41b90e86305c6270917265/scratch/geoffrey/y2023/sql_basic.cljc#L15C17-L15C17#2023-07-0212:12Dustin Getzthat sounds like the same hack to me#2023-07-0212:13Dustin Getzyou can propagate the time value by electric dynamic scope , doesn’t that reduce the boilerplate down to like 2 characters?#2023-07-0212:13Dustin Getzyou can also use partial on a wrapped query function and bind that wrapped-q in dynamic scope#2023-07-0414:23Adrian Smithhttps://twitter.com/dustingetz/status/1676220936440225794 🙂#2023-07-0512:48VincentAwesome.#2023-07-0414:31JHi! What is hypperfiddle.popover? It’s like a modal?#2023-07-0414:32Dustin GetzThat's an experimental thing that is part of our "Crud Spreadsheet". The popover in master that you're looking at is integrated with Datomic. The next iteration of it (not yet released) works differently and is not integrated with Datomic#2023-07-0414:34Dustin GetzYes its an abstraction over a modal form however it operates at a very high level, i dont think it is useful to electric users today#2023-07-0414:49JThanks @U09K620SG. In the future, electric will have a base components toolbox? Like modal , switch, etc...#2023-07-0414:51Dustin GetzOur priority is to release robust UI controls and design patterns that people can easily extend and build on top of#2023-07-0414:52Dustin GetzYes, as we build reusable components for our customers we will release them as we can#2023-07-0414:57JWonderfull!#2023-07-0507:28JHi! I want to trigger an event only when a click occur on a parent div not on the child. In vanilla JS there is a trick for that but didn’t work on the cljs/electric land. Here the example:
(dom/div
(dom/on "click" (e/fn [e] ;; should fire only when this div is clicked not the child))
(dom/div ...)
#2023-07-0507:31JI had an id on the parent div and check the e.target.id when the click occur.#2023-07-0507:57s-olregular JS rules apply, you could:
• attach another "click" handler on the child and call (.stopPropagation e)
• compare (= (.-currentTarget e) (.-target e)) (see https://developer.mozilla.org/en-US/docs/Web/API/Event/currentTarget) #2023-07-0507:57s-olsetting an id or any other attribute and checking that should work as well#2023-07-0507:58JThanks @U05AL1ZH8TW. At the beginning, I wanted to check the -currentTarget but in the event is set to null#2023-07-0507:59s-olhm, that is odd. electric doesn't have a virtual DOM or virtual events or anything of the sort afaik, it's just plain JS so any weirdness you see should be related to DOM and browser#2023-07-0508:01JYes it’s weird#2023-07-0512:28Geoffrey GaillardFrom MDN, maybe it will help:
> Note: The value of event.currentTarget is only available while the event is being handled. If you console.log() the event object, storing it in a variable, and then look for the currentTarget key in the console, its value will be null.
> Instead, you should either use console.log(event.currentTarget) to be able to view it in the console or use the debugger statement, which will pause the execution of your code thus showing you the value of event.currentTarget.#2023-07-0516:32braai engineerWhile porting my Electric app from XTDB to Datomic Pro, I encounter this exception when I try to use e/offload:
Unable to resolve classname: (hyperfiddle.electric/-offload (fn* [] (range 10)) missionary.core/blk)#2023-07-0516:33Dustin Getz~:require missionary to work around, that looks like a macro error on our part~#2023-07-0516:33Dustin Getzhmm i dont see a coding error at first glance#2023-07-0516:34Dustin GetzOh, are you using e/offload in a e/client?#2023-07-0516:34Dustin Getzit's e/server only#2023-07-0516:35Dustin GetzI just captured a ticket to improve that error essage#2023-07-0516:35braai engineer(am using on server) Hmm I think it's a bug in my program because if I swap out the offloaded function for (fn [] []) it works#2023-07-0516:36braai engineercould it have something to do with Datomic db being GC'ed before it runs or because it runs on different thread?#2023-07-0516:37Dustin Getzno i dont think so#2023-07-0516:37Dustin Getzbased on the error, the e/offload macroexpands to `(new (-offload ~f! m/blk)#2023-07-0516:37Dustin Getzthe new is treating the parameter as a class#2023-07-0516:38Dustin Getzthat's really odd, i'll have to check with @U09FL65DK#2023-07-0516:40Dustin Getzthe macro is really simple though I dont think this issue is a bug in offload, but nor can i perfectly explain the error#2023-07-0517:00braai engineerSo this is some weird behaviour that's hard to debug because of null pointer error.
If I take the database out of the offload and just call it direct, same thing happens (so it's not related e/offload):
(lev [{:as results
:keys [limit offset
query-fn count-fn]} (data/fetch-txes db filters)
total-count (e/offload count-fn) ;; this works fine
results (query-fn)] ;; this works in REPL but throws in Electric:
ERROR hyperfiddle.electric-jetty-adapter: Websocket error
java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
I can run it in the REPL just fine, but as soon as it's in Electric, it blows up. The same approach worked fine for XT, but breaks for Datomic.#2023-07-0517:09braai engineerOK, figured it out (my bad). Something lower down was trying to treat the result (a vector of Longs) as a vector of maps, so the error was not related to calling (query-fn), but lower down. Hard to tell though#2023-07-0517:14braai engineerPossible to get a more accurate line number for these kinds of errors?
Somewhere in a component it knows that I'm trying to reference a symbol account (a common symbol in my codebase), and it would help if it told me where.#2023-07-0517:16Dustin Getzyes but not until after the next major version lands#2023-07-0517:17Dustin Getzincremental compilation is going to change everything #2023-07-0517:17braai engineervery excited for incremental compilation#2023-07-0517:18braai engineerI sent my colleague (a JS guy's who's only ever worked at this employer) your talk and this is what he said:#2023-07-0518:57nakkayaWhat would be the flow for say when a button is pressed a modal is shown? From a button on the side bar I would like to show a little form in a modal? Do I need to trigger a redraw of the whole page and add the form there or is there a better way?#2023-07-0519:10Dustin Getzuse a boolean in an atom for the open/closed state, use an if statement to draw the open modal, use css to break out of the organic layout#2023-07-0519:11Dustin Getzwe have some css you can use, not at my desk so i’d have to track it down later- it’s in the electric repo, popover.css or something. also see popover.cljc as an example (you can’t use this one for reasons explained in a thread a few days ago but it shows how to accomplish it)#2023-07-0519:13Dustin Getzyou’re going to need more atoms and callbacks to coordinate the form state reagent-style for now, we have a much better signals based pattern coming but it’s not ready yet unfortunately#2023-07-0519:43nakkayathanks I remember the thread. Will check popover.cljs#2023-07-0520:09Dustin Getzyou can also integrate a third party popover, i looked into it last year and some are very high quality, iirc some are css only which would be trivial to integrate. the reason we ultimately did our own is because it was like 10 lines of css for the basics#2023-07-0520:09Dustin Getzwhat the third party libs will get you is the auto layout with window clipping detection#2023-07-0521:40braai engineerIs there a rule of thumb for performance when displaying a list of 50-100 entities to decide between datomic.api/pull-many for pulling multiple entities at once vs. d/pull per each row? (I haven't benchmarked) (Datomic Pro)#2023-07-0521:46chaosHI, is it possible to incorporate HTML code directly on the client side? e.g. SVG Icons, without the need to convert them to hyperfiddle.electric-svg calls first, thanks#2023-07-0522:41Vincentyou can always edit index.html , provide style in the head tags and selectively alter css properties in your app.#2023-07-0600:34Dustin Getz@U012BL6D9GE can you be more specific #2023-07-0600:37Dustin Getzyou could write a macro that loads SVG templates from the file system and attaches them using innerHTML point effect, would that solve your problem?#2023-07-0600:38Dustin Getzsee impl of electric-dom and electric-svg it’s not hard #2023-07-0600:38Dustin Getzor in this case a svg file#2023-07-0606:45chaosgreat, thanks I came up with the following example. I assumed here that dom/node refers to the parent node
(let [icon "<svg xmlns=\"\" fill=\"none\" viewBox=\"0 0 24 24\" stroke-width=\"1.5\" stroke=\"currentColor\" class=\"w-6 h-6\">
<path stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M4.26 10.147a60.436 60.436 0 00-.491 6.347A48.627 48.627 0 0112 20.904a48.627 48.627 0 018.232-4.41 60.46 60.46 0 00-.491-6.347m-15.482 0a50.57 50.57 0 00-2.658-.813A59.905 59.905 0 0112 3.493a59.902 59.902 0 0110.399 5.84c-.896.248-1.783.52-2.658.814m-15.482 0A50.697 50.697 0 0112 13.489a50.702 50.702 0 017.74-3.342M6.75 15a.75.75 0 100-1.5.75.75 0 000 1.5zm0 0v-3.675A55.378 55.378 0 0112 8.443m-7.007 11.55A5.981 5.981 0 006.75 15.75v-1.5\" />"]
(e/client (dom/div
(-> dom/node
(.-innerHTML)
(set! icon)))))
#2023-07-0610:44Dustin Getzyes and you can do (defmacro inline-file [filepath] (slurp filepath)) — note no backtick — to load that string and bake it at compile time#2023-07-0610:45Dustin Getz(from memory )#2023-07-0600:49Dustin GetzI’m talking and doing beginner/intermediate Q&A at the Clojure visual tools meet up on Friday https://clojurians.slack.com/archives/C02V9TL2G3V/p1684446434308189#2023-07-0712:25Daniel SlutskyHi. Here is a reminder of the Electric meetup later today. ☝️
https://clojureverse.org/t/visual-tools-meeting-19-electric-login-for-details/
If you are planning to attend, it would be helpful to comment on @dustingetz in's questions in the linked #visual-tools thread.#2023-07-0712:26Dustin GetzI'm prepping right now, appreciate any thoughts!#2023-07-0712:37Daniel Slutsky@dustingetz a little thought:
Even though this is an informal, unstructured session, if you wish we can record it.
Then, later you can decide if you wish me to extract some parts and share them publicly.#2023-07-0712:50Dustin GetzI'd prefer not to share any recording publicly, we like to publish only the highest quality rehearsed content at this time. Nonetheless certainly it might be helpful to have the recording for internal use so if it's easy for you to do, that would be helpful#2023-07-0712:51Daniel Slutsky👍#2023-07-0712:51Dustin GetzAnd you're right, maybe we get lucky with a segment being worth sharing#2023-07-0715:58Daniel SlutskyThe Electric meeting is starting in a few minutes.
https://bit.ly/data-recur-zoom#2023-07-0818:39braai engineerSorry I could not make this. Hope there is a recording even if it's not public 🙂#2023-07-1316:18grounded_sageOh I wanted to do this but missed it. Also hope there is a recording accessible. #2023-07-0606:26JHi! I made a macro to convert any svg file in electric component. Feel free to use it (https://gist.github.com/jeans11/757617c45801febae618e75e1581a953) . Thanks to the heroicons-electricrepo for the help to create the macro that use the electric component ^^.#2023-07-0606:28JI use it with https://iconoir.com/#2023-07-0606:38tobiasNice! Is there an advantage to doing things this way rather than set the inner HTML of an element?#2023-07-0606:49JWith the set inner html you loose the ability to easly custom the svg, no?#2023-07-0607:43Geoffrey GaillardDepends on the use case. Converting SVG into electric is most of the time a little bit too much, setting innerHTML is most of the time not enough.
https://betterprogramming.pub/the-best-way-to-use-svg-icons-in-your-website-ea8312b8f07a#2023-07-0607:49Geoffrey GaillardThank you for the gist, this is an improvement over the heroicons-electric's macro. Feel free to submit a PR :)#2023-07-0607:58JThanks! On the heroicons-electric repo directly?#2023-07-0607:58Geoffrey GaillardYes, https://github.com/hyperfiddle/heroicons-electric#2023-07-0609:36tobiasYou mean because if you set inner HTML then you can't style with CSS?#2023-07-0612:10Geoffrey GaillardCSS works. But you don’t control the child nodes since all you have is a string. E.g. setting attributes.#2023-07-0612:16tobiasIf you get the svg icon as an electric component (using the svg->electric macro from the gist shared above) then how do you set the attributes of child nodes? Apologies for the silly question I'm probably missing something really obvious.#2023-07-0612:20JI make this macro to consume existing icons. I not need to custom the attribute childs. However, I want customise the color, width, height of the svg. This is the props that the svg element can take.#2023-07-0616:09Dustin GetzI want to sharpen the tradeoffs around the three approaches:
• phrase the SVG entirely in Electric
• phrase the SVG in Missionary
• phrase the SVG as a string and set innerHTML
Electric is about e/server e/client and therefore Electric code pays overhead to make that possible. Since an SVG element will likely never use network transfer, using Electric to manage a reactive and networked DAG is "overkill" and could become a performance or memory issue, especially if there are hundreds of nested elements or attributes.
Missionary is about low-level reactive programming with explicit backpressure control and explicit memoization/allocation. If computation needs fine-grained reactivity but not network transfer, Missionary will be faster because it doesn't pay the overhead of a distributed runtime with distributed "stack frames". There is still some overhead for the reactive propagations – at it's core, reactive programming is a time/space tradeoff, we save time by reusing cached results.
Set innerHTML is going to be fast and cheap, as it is a single write to a javascript reference (assuming the SVG has no reactivity points). If the SVG has reactive attributes or other reactivity points we might express exactly those reactive boundaries using Missionary.#2023-07-0616:12Dustin GetzObviously, for early stage projects it makes no difference, and doing things entirely in Electric is fine until demonstrated otherwise. I think we likely would start notice a perf hit asking Electric to manage several hundred dom elements or attributes, which electric-dom will turn into N point writes performed in rapid succession. A rule of thumb is the average webpage has up to 1-2k dom elements total; and it is mostly a tree hierarchy#2023-07-0616:33JThanks @dustingetz for the explanations. How make an svg with missionary? :thinking_face:#2023-07-0616:46Dustin GetzHa it would be syntactically horrible, probably you would separate the static parts into one big string template, and then handle the reactive dynamic parts with individual set!#2023-07-0618:07Dustin GetzCorrect but electric still maintains inner machinery related to network#2023-07-0618:31Toyam CoxHi, I've been searching the backlogs of this channel, but unfortunately couldn't find what I wanted... I'm trying to mount a ring handler to return electric. I get that I need to return a page (regular html) which sources the electric bundle, but how do I also mount the electric bundle on the server side? I'm trying to add a dynamic page with electric to an existing application#2023-07-0618:32Toyam CoxI think I need to add ring middleware? How do I tell the middleware to listen under a path for the websocket init?#2023-07-0618:33Dustin GetzThe starter app contains a functioning server https://github.com/hyperfiddle/electric-starter-app/tree/main#2023-07-0618:33Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/16fff5da32b0a1f50d221cba56c872632ee10d2d/src/prod.clj#L13#2023-07-0618:34Toyam CoxI've seen it https://github.com/search?q=repo%3Ahyperfiddle%2Felectric-starter-app%20ring&type=issues
and I need to attach this to an existing server.#2023-07-0618:35Toyam CoxI've already got a run-jetty and I want to introduce electric as a single part of this bigger application#2023-07-0618:35Toyam Coxease-in as it were#2023-07-0618:37Dustin GetzThe best I can do right now is point you to the electric-websocket-middleware in the Jetty10 example, which might be closer to what you are looking for than the Jetty9 example https://github.com/hyperfiddle/electric-starter-app/blob/16fff5da32b0a1f50d221cba56c872632ee10d2d/src/electric_server_java11_jetty10.clj#L115#2023-07-0618:48Dustin GetzWe also have an unpolished fulcro starter app as of this week, not sure if it is helpful, it contains a HttpKit middleware example and a fulcro component that connects to the electric server when it mounts https://github.com/hyperfiddle/fulcro-electric-template/tree/master/src/main/app/server_components#2023-07-0619:01Toyam CoxThanks, I'll poke at this and see if I can make it work for the application#2023-08-2222:01Toyam CoxAlright, I am finally getting the time to do this (Mounting electric at a point inside an existing application). Just checking to see if there are new developments in the last 2 months before I go learn how the bundle is built and exposed to the app 🙂#2023-08-2222:06Toyam CoxI'm grabbing the latest starter apps and trying to graft it into a minimal "existing application" to see what it takes#2023-08-2222:23Dustin Getzthe ring alpha release supports websockets so might make this a lot more straightforward, but we haven’t had time to take a look#2023-08-2222:23Dustin Getzhow much effort do you intend to put into this?#2023-08-2222:31Toyam CoxI intend to try and make it work repeatedly over the next 2 weeks. It would be really cool to help me unblock on a few things.
Ring Jetty supports websockets too#2023-08-2302:33chromalchemyYou might want to check out this integration with Biff framework. It is an example of adding Electric within a pre-existing app with ring middleware.
https://biffweb.com/p/how-to-use-electric/#2023-08-2511:10Toyam CoxI'm at the point of working on compilation without a shadow-cljs project set up. The point is to incorporate electric in an existing app, so this is one of the steps along the way in my mind#2023-08-2511:10Toyam Coxit's bad that I've had the Biff article up for like a month now I think#2023-08-2511:11Toyam Coxthe goal is to highlight what build options are actually needed#2023-08-2511:11Toyam Coxit might only be possible to mount electric once in an app (one entry-point in an existing project)#2023-08-2511:18xificurCthe paved path is clj server, cljs client, shadow-cljs to compile the client, single e/boot call to start up electric. There's a swarm of other options that should work just as well (e.g. multiple entrypoints, clj client, non-shadow build) but we haven't tried it out.
What are you trying to achieve?#2023-08-2511:21Dustin GetzI also don't understand (beyond integrating the electric websocket middleware with ring, that i understand)#2023-08-2704:48Toyam Cox@U09FL65DK I have an existing MPA on clojure. I want to get electric for a new feature on that MPA. Thus I need to, with the least amount of surprise, make electric end up hosted instead of being the king of the serverside#2023-08-2704:53Toyam CoxI'm getting closer!#2023-08-2704:55Toyam CoxI'm doing it in a small replication example that I will open source as soon as it works#2023-08-2706:31Toyam Coxthough I have learned I can use shadow-cljs without not-using lein, so that's something I'll be trying (since cljsbuild doesn't build, since *env* isn't defined [first error, no idea as to root])#2023-07-0712:25Daniel SlutskyHi. Here is a reminder of the Electric meetup later today. ☝️
https://clojureverse.org/t/visual-tools-meeting-19-electric-login-for-details/
If you are planning to attend, it would be helpful to comment on @dustingetz in's questions in the linked #visual-tools thread.#2023-07-0715:58Daniel SlutskyThe Electric meeting is starting in a few minutes.
https://bit.ly/data-recur-zoom#2023-08-2222:01Toyam CoxAlright, I am finally getting the time to do this (Mounting electric at a point inside an existing application). Just checking to see if there are new developments in the last 2 months before I go learn how the bundle is built and exposed to the app 🙂#2023-07-0701:58Vincenti have a style question. when should i wrap a clojure-only function with the #?(:clj reader conditional?
is it only necessary to wrap #?(:cljs statements?#2023-07-0712:44Dustin GetzThe reader conditional is needed to guard a piece of code in a CLJC file that would otherwise fail to compile. Whether you put the conditional around the function definition, or scope it narrowly around the illegal expression, is up to you.
There are a couple gotchas, e.g.:
(defonce !conn #?(:clj (d/create-conn {}) :cljs nil)) Here, in defonce the value position is mandatory, so if we omit the :cljs branch it would fail to compile in :cljs
#?(:clj (defonce !conn (d/create-conn {}))) This is fewer characters, so I prefer it
Semantically, the former creates !conn twice (in both the JVM and the JS namespace), and the latter creates !conn only on the server. However, it makes no technical difference to have that extra unused var. In our codebase you see all styles because at one point the Electric compiler needed to resolve the symbol on both CLJ and CLJS even if the symbol was not actually used in CLJS. With that long fixed, now my rule of thumb is to prefer the more concise phrasing.
I recommend not getting hung up on indentation and whitespace. Basically just accept that the reader conditionals will damage the cute and consistent top-level indentation that we are accustomed to from Clojure. For example https://github.com/hyperfiddle/electric/blob/9833e053aa2a12c1234d91c450fbf9f8ddec457b/src/hyperfiddle/electric.cljc#L44-L52. Nothing can be done about it. In that file you'll see we use many different styles - case by case we do our best. Broadly we try to reduce LOC while also keeping line width down.#2023-07-0712:46Dustin GetzNote: Electric does not support reader conditionals due to limitations of clojure, you'll need to wrap them in an ordinary clojure/script function and call it from electric#2023-07-0714:06telekidI've been trying to build my intuition around the "UIs are streaming DAGs" notion. I found the game play metaphor from the 10 minute overview to be particularly helpful, but I still left uncertain about how exactly it is that the client-side and server-side code orchestrate their relationship, particularly with regards to event ordering. That is, how do the client and server behave as if they are executing a single, well-ordered program? After sitting on it for a bit, this is what I've come up with:
1. Lexically, programs are obviously ordered. Memory is ordered, files are ordered, tokens are ordered, etc.
2. Similarly, an order can be imposed upon an AST. What I'm really talking about here is an evaluation evaluation strategy, something that implies an evaluation order. One could imagine many different evaluation strategies for a given AST.
3. It is at the boundary between syntax and semantics that asynchronicity is introduced. In the case of electric, missionary describes a particular set of semantic interpretations of syntactic structures that allow a value (something that can be bound to a variable) to express a program fragment that has a well-defined (if not deterministic) ordering.
4. Using the notions of a single evaluation order for the AST and the unambiguous semantics of asynchronicity-as-a-value, the electric compiler can basically rewrite the program into two (theoretically more, I imagine!) components. These components can execute concurrently. Variable bindings act as computational edges that may happen to cross a network boundary, but since bindings are associated with values, and missionary values encapsulate well-defined orderings, the outcome is a single "program" with an ordering that is also well-defined.
Does this make sense? Am I missing anything important here, or getting anything wrong?#2023-07-0714:29Dustin GetzYes, the Clojure syntax is translated into Missionary flow syntax, the missionary propagation algorithm guarantees a glitch-free ordering with respect to which minimum set of edges must recompute when an upstream input changes. During the Clojure to Missionary translation we also add in instrumentation for the network planner. Edges that cross network have IO/latency, but note that all inputs (like a dom event) originate from IO. Due to latency there is not a globally defined order, but locally within each of the two sites there is a local ordering, and at each site Missionary has authority over its local order, so in the event of a conflict (e.g. server sent a reply but the client no longer wants it) it can be resolved automatically, it is clear who has authority. Today the sites broadcast their interests – imagine a sports team captain yelling a command to change the play on the fly, the players reconfigure. Here we have two captains, client and server. The change commands can all be validated since the playbook is coordinated ahead of time.#2023-07-0716:14Vincentthere's also the "Pending" state, which makes it all make more sense for me 😅#2023-07-0716:45telekidThank you both! Very helpful. Makes sense.#2023-07-0719:08Dustin Getzquick Architecture summary https://gist.github.com/dustingetz/a68b4b19b2818f51b46d8111966b8c7b#2023-07-0809:36teodorlu> PG as a query planner to choose the fastest query
>
What's PG?#2023-07-0812:11Dustin Getzpostgresql#2023-07-0814:07teodorluHuh, interesting. PostgreSQL was the only PG i could think of - but I couldn't imagine that PostgreSQL's query planner could be used inside Electric!#2023-07-0814:07Dustin Getzit cannot, sorry#2023-07-0814:07Dustin Getzjust an example#2023-07-0815:10teodorluAh, ok 👍#2023-07-0719:14srcererThanks for talking about Electric. Truly amazing work!#2023-07-0720:45Daniel Slutskyhttps://clojurians.slack.com/archives/C02V9TL2G3V/p1688762698805309#2023-07-0815:54Reut SharabaniHow can I invoke some callback after an element is actually loaded?
I'm trying to use mermaid to render some content and it requires calling (.run js/mermaid (clj->js {:querySelector ".mermaid"})) after the mermaid class has content (which is fetched from the server).
I wonder how I can know when the content was actually generated since there are no elements that support onload in dom/* .
I tried stuff like:
(when (not (s/blank? mmd))
(try
(dom/div (dom/props {:class "mermaid"})
(println "rendering" mmd)
(dom/text mmd))
(finally (.run js/mermaid (clj->js {:querySelector ".mermaid"})))))
Or:
(when (not (s/blank? mmd))
(dom/div (dom/props {:class "mermaid"})
(println "rendering" mmd)
(dom/text mmd)
(.run js/mermaid (clj->js {:querySelector ".mermaid"}))))
But it seems run (which returns a promise) still happens before the content is actually loaded.#2023-07-0815:58Dustin GetzWhen you say "it [content] is actually loaded" what exactly are you referring to as "content"#2023-07-0816:00Reut Sharabanithe class should exist and should contain the mmd.
I should have mentioned mmd is a reference (?) it is defined as a watch (?) because it is communicated from the server.
To give the overall perspective I have mmd files (mermaid files) on the server and I want to display them on the client. The client cannot download them directly via links, so I'm using a reference (atom + watch) by looking at examples.
This is the definiton of mmd:
(e/server
(let [!mmd (atom "")
mmd (e/watch !mmd)]#2023-07-0816:00Reut Sharabani(mmd is just a string)#2023-07-0816:02Dustin Getz(let [x (e/server "foo")] (case x (js/mermaid.run ...)))#2023-07-0816:03Dustin Getzthis is an idiom we currently use that exploits the fact that the conditional node cannot evaluate it's body until after the remove value is available, i.e. if x is pending the control flow is blocked until we can test it#2023-07-0816:03Dustin Getz(here we are using the default branch)#2023-07-0816:04Reut SharabaniI'll try and see what you mean#2023-07-0816:04Reut Sharabanioh I think I get it actually, sec#2023-07-0816:05Dustin GetzIt would be better if you could write (js/mermaid.run (e/server ...)), where the Electric evaluation rules will delay the .run until the parameter is available#2023-07-0816:05Reut Sharabaniisn't my when (in the original message) doing exactly that though?#2023-07-0816:05Dustin Getzhmm yes#2023-07-0816:05Reut Sharabanithe mermaid/run method receives selector (among other things) to render the matching elements. It does not receive mermaid strings.#2023-07-0816:05Dustin GetzCan you post a minimal (complete) example that runs#2023-07-0816:06Reut Sharabaniyes#2023-07-0816:06Reut Sharabanilet me edit it to be minimal, it can take a while#2023-07-0816:08Dustin Getzoh, i think what's happening is that mmd starts "" and then later receives a value? in this case js/mermaid.run does not have a dependency on mmd and therefore Electric will not re-evaluate it if it has already ran#2023-07-0816:10Dustin GetzIt is idiomatic to wrap it like (defn mermaid! [_mmd] ...) to force the dependency explicitly#2023-07-0816:10Reut Sharabani(e/defn Observer []
(e/server
(let [!mmd (atom "flowchart LR\n\nA-->B")
mmd (e/watch !mmd)]
(e/client
(.initialize js/mermaid {:startOnLoad true})
(when (not (s/blank? mmd))
(try
(dom/div
;; doesn't work
(dom/div (dom/props {:class "mermaid"})
(println "rendering" mmd)
(dom/text mmd))
;;works
(dom/div (dom/props {:class "mermaid"})
(println "rendering" mmd)
(dom/text "flowchart LR\n\nA-->B")))
(finally (.run js/mermaid (clj->js {:querySelector ".mermaid"})))))))))#2023-07-0816:11Reut Sharabanithe try-finally is just an attempt it could not make sense at all 🙂#2023-07-0816:11Dustin Getzyeah i dont think the try/finally is helping you here#2023-07-0816:11Reut Sharabanijust wanted to write the run somewhere#2023-07-0816:12Reut SharabaniI think creating a dependency worked, checking on the "real" code now#2023-07-0816:18Reut SharabaniThis worked:
(defn mermaid! [_mmd]
(.run js/mermaid (clj->js {:querySelector ".mermaid"})))
And then:
(when (not (s/blank? mmd))
(try
(dom/div (dom/props {:class "mermaid"})
(println "rendering" mmd)
(dom/text mmd))
(finally (mermaid! mmd)))))))))#2023-07-0816:18Dustin Getzwhy try/finally#2023-07-0816:18Reut Sharabaniwhere would you invoke mermaid! ?#2023-07-0816:18Dustin Getzinvoke it like println#2023-07-0816:19Reut Sharabaniwould it work coincidentally because it's a promise?#2023-07-0816:19Dustin Getzi dont understand#2023-07-0816:20Reut Sharabanidoesn't println happens before dom/text is generated?#2023-07-0816:20Dustin Getzah#2023-07-0816:20Dustin Getz(dom/div
(do
(dom/props {:class "mermaid"})
(dom/text mmd)
(js/mermaid.run #js {:querySelector ".mermaid"})))
#2023-07-0816:20Dustin Getzi have added the explicit do to clarify what is happening#2023-07-0816:20Reut Sharabaniit throws some warning like that#2023-07-0816:20Reut Sharabanilet me re-check#2023-07-0816:21Reut SharabaniUnserializable reference transfer: #object[Promise] [object Promise]
#2023-07-0816:21Reut Sharabaniit's not really generating anything dom-able#2023-07-0816:21Reut Sharabaniso I don't like it#2023-07-0816:21Dustin Getzi dont understand what you don't like?#2023-07-0816:22Dustin Getzwhat don't you like and why#2023-07-0816:22Reut SharabaniIt gives me this warning on the repl:
Unserializable reference transfer: #object[Promise] [object Promise]
#2023-07-0816:22Dustin Getzthe warning can be ignored, i will explain that in a moment, it is harmless, i will also explain how to fix the warning#2023-07-0816:23Reut Sharabaniok, but I don't want to take too much of your time#2023-07-0816:23Dustin Getzi want to know "what don't you like and why"
Its just the warning that concerned you?#2023-07-0816:23Reut SharabaniI can mess with it some more and learnt he conventions later#2023-07-0816:23Reut Sharabaniyeah#2023-07-0816:24Reut SharabaniI think the warning is based on it not being embeddable in the dom (returns promise)#2023-07-0816:24Reut Sharabanibut that's just intuition#2023-07-0816:24Dustin Getzthat's not it#2023-07-0816:24Dustin Getzthe warning is because the promise is returning up the "stack" into an e/server block#2023-07-0816:24Reut SharabaniI have no idea what that means#2023-07-0816:25Reut Sharabaniif there is a link feel free to direct me there and let me learn the hard way 🙂#2023-07-0816:25Reut Sharabanibut I'm very focused on making things work so I can show it at work tomorrow#2023-07-0816:27Dustin Getz(e/server (e/client (js/Object.))) will warn because the reference cannot be serialized with transit to move over network to the server#2023-07-0816:27Dustin Getzonly values can move over network, promise is a ref type and cannot be serializable#2023-07-0816:27Reut SharabaniI see#2023-07-0816:28Dustin Getz(e/server
(let [x (e/client
(dom/div
(dom/props {:class "mermaid"})
(dom/text mmd)
(js/mermaid.run #js {:querySelector ".mermaid"})))]
x))
x is the promise and it will fail to transfer#2023-07-0816:28Reut Sharabanioh now I understand what you said about the stack#2023-07-0816:28Reut SharabaniI think#2023-07-0816:29Dustin Getzthe dom elements wrap their children forms in an implicit do (for effect) and return the final element#2023-07-0816:29Reut SharabaniI have another meeting now Dustin, but I will continue my exploration later#2023-07-0816:29Reut Sharabanithanks for your help!#2023-07-0822:07Reut SharabaniThe communication (through datascript) is just incredible!
I can't imagine building an api this specific so easily in something like re-frame and rest.
Can't wait to test this as an actual service to see how performant it actually is.
I built a small ui to see the data flow through the services in the company that employs me. It's not done but I'm getting the hang of it.#2023-07-0822:14Dustin Getzplease report back your findings!#2023-07-0915:58Reut SharabaniFor future reference to those who search in slack:
Seems like mermaid dynamically updates the dom in a way electric doesn't like.
It appends an svg to the dom (under the mermaid class) and then (for me) it stopped responding (changing mmd doesn't change the dom anymore).
I ended up doing it manually like this:
(e/defn Mermaid []
(e/client
(try
(dom/div (dom/props {:class "mmd"})
(dom/div (dom/props {:id "mermaid"})
(dom/text svg)
(.then
(.render js/mermaid "dummy" mmd)
(fn [svg] (let [e (.getElementById js/document "mermaid")]
(println "element" e)
(println "svg" (.-svg svg))
(set! (.-innerHTML e) (.-svg svg)))))
nil))
(finally
(mermaid! mmd)))))
There are probably better solutions but I don't want to figure it out right now.#2023-07-0816:25grounded_sageIs there a way to use raw html. Say there is an avg I want to use as pure code instead of pulled in as an image. #2023-07-0816:26grounded_sageI can also convert all to the tags to ‘(dom/…)’ I was just wondering if there was a dirtier faster way#2023-07-0816:30Dustin Getzset innerHTML on a string#2023-07-0816:30Dustin Getzor use a svg file, surely there is a way#2023-07-0820:52Vincenti'm a little sad i missed the zoom meeting, i forgot y'all live in London Lol#2023-07-0902:44braai engineerHow to listen for dragstart and dragend DOM events? (dom/on "dragstart" ...) or "ondragstart" do not seem to fire.#2023-07-0913:27Dustin Getzif you suspect an issue in electric-dom with these events you can drop down to m/observe and manage the event directly#2023-07-1409:29braai engineerI am not yet skilled enough in Missionary to know how to drop-in to it from Electric. Is there an example somewhere for a different event plz?#2023-07-1410:42braai engineerAh, my mistake: I was not calling (.preventDefault e) on "drop" and "dragover" events.#2023-07-0918:25VincentI have a need for a component I think everyone could employ eventually in one project or another:
a way to pass a timestamp to the clientside and have it say in relative terms "1 second ago" or "15 minutes ago" or "1 hour ago"
it makes me think that there are probably a few we could optimistically invest our collective time into as a community and make it easy to have durable, reusable components#2023-07-0919:22Dustin GetzJavascript ecosystems have it covered - https://day.js.org/docs/en/display/from-now#2023-07-0919:50Piotr Roterskiif you want to stay in cljc land you could probably use https://github.com/clj-commons/humanize#datetime#2023-07-0918:43VincentFree multiplayer and client/server data transparency is a revolution in web programming. It's hard to state how liberating this is for applications that rely on real-time comms or significant client/server message passing.#2023-07-0919:49Reut SharabaniWhat is HYPERFIDDLE_ELECTRIC_SERVER_VERSION ?
Why do I need to set it?
How do I set it?
Can I not set it/ignore it somehow and still have everything work?#2023-07-0919:50Reut SharabaniI'm trying to deploy to production and I have issues. One of these issues is that this property isn't set. I set it manually and now I suspect the other issues may be related to not setting it properly (?)
I just set it to my commit hash because I have no idea what it should be.#2023-07-0919:52Reut SharabaniI'm deploying to an internal env and version is not set and the docker cmd is:
"Cmd": [
"/bin/sh",
"-c",
"java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$VERSION -jar app.jar"
]
Can I safely change it to java -jar app.jar?#2023-07-0920:07Reut Sharabanifor now it's set to my commit hash and I get these errors 😞 :
in ( .render js/mermaid dummy )
in ( .then (.render js/mermaid "dummy" mmd) #object[ja] )
in reactive (fn [] ...)
in reactive (fn [] ...)
in reactive (defn Mermaid [] ...) in app/observer.cljc line 233
in reactive (fn [] ...)
in reactive (fn [] ...)
in reactive (defn Observer [] ...) in app/observer.cljc line 264
in (try ...)
TypeError: is not a function
Locally it works fine so it's either something to do with the remote env or with my build. I have no idea what js/mermaid is available on the webpage (checked in console) and it has a render method.#2023-07-0920:13Dustin Getzthe starter app has a working configuration out of the box, are you having trouble with that?#2023-07-0920:14Dustin Getzthe version used to cachebust stale clients when you deploy and the server changed#2023-07-0920:15Dustin Getzit should be set by the build, i think in the dockerfile#2023-07-0920:17Dustin Getzsee build.clj and the readme #2023-07-0920:18Reut SharabaniI'm not deploying to that fly platform thing. But I don't think it's the version. I think it's because I set a callback on a promise using (fn [arg] ..) and it doesn't recognize it as a function. But I'm still investigating#2023-07-0920:19Reut Sharabanithis is my clue:
in ( .then (.render js/mermaid "dummy" mmd) #object[ja] )
in reactive (fn [] ...)
in reactive (fn [] ...)
in reactive (defn Mermaid [] ...) in app/observer.cljc line 233
in reactive (fn [] ...)
in reactive (fn [] ...)
in reactive (defn Observer [] ...) in app/observer.cljc line 264
in (try ...)
TypeError: is not a function
at Function.<anonymous> (main.BC569D98165855497A21AC1C8BE1718D.js:1623:185)
The object thing is a ja and the http://ja.mg is not a function (which is probably function/call or something)#2023-07-0920:19Dustin Getzwhat commands did you type? where is the error coming from?#2023-07-0920:21Reut SharabaniIt's my app running and trying to update some svg. The library that generates the svg is using async/away (so a promise) and my line that probably fails is:
(.then
(.render js/mermaid "dummy" mmd)
set-mm-svg)
set-mm-svg used to be a lambda (fn [svg] ... set svg) and I think that failed#2023-07-0920:21Dustin Getzrelease build? dev build?#2023-07-0920:21Reut Sharabanirelease (prod)#2023-07-0920:21Reut SharabaniI also tried e/fn and it failed so I am now trying #?(:cljs (defn ...))#2023-07-0920:23Dustin Getzok, i think it’s an advanced mode problem, i think you don’t want to access mermaid from the js global. how is mermaid loaded?#2023-07-0920:23Reut Sharabaniindex html script tag#2023-07-0920:23Reut SharabaniI have very little front end experience 😄#2023-07-0920:24Reut SharabaniIt's available on the webpage (I checked in console)#2023-07-0920:24Dustin Getzthat is the problem #2023-07-0920:24Reut Sharabaniwell after 3 hours that is nice to hear 🙂#2023-07-0920:24Dustin Getzget mermaid from NPM and then :require it so that shadow-cljs can see it#2023-07-0920:25Reut Sharabaniyeah I wanted to do it but idk how require works in clojurescript. I guess I'll have to do that#2023-07-0920:25Reut Sharabaniit's just adding to the shadow under :dependencies, right?#2023-07-0920:25Dustin Getzidk off hand and am on mobile#2023-07-0920:25Reut Sharabanino problem, let me try. Thanks Dustin!#2023-07-0920:26Dustin Getzyou’re definitely going to need to have working knowledge of clojurescript toolchain#2023-07-0920:26Reut Sharabaniwell I'm stubborn so I'll definitely learn /something/#2023-07-0920:27Reut Sharabanibtw why does the deployment care, does it modify index.html?
If there is a link where I should have found out I'd like to read. Thanks.#2023-07-0920:29Dustin Getzclojurescript prod builds perform dead code elimination and code minification which mangles the names#2023-07-0920:30Dustin Getzyou can also turn it off #2023-07-0920:31Reut SharabaniI wanted to. I only found optimizations and turned them off#2023-07-0920:31Reut Sharabani(It didn't work so they're on now).#2023-07-1009:41Reut SharabaniI couldn't get it to work because shadow seems to not support "lazy loading" which is probably when you require something conditionally and I end up with some missing modules after compilation.
I think it's best if I can add the html "script" tag and not mangle anything because performance is not an issue for me.
Is turning off "optimizations" enough to do that or do I need to take care of other things?
If it's too much work, how can I include a library with lazy loading (in this case mermaid) and use it in production?
It works locally so there must be a difference in that aspect between production and local build but I've yet to find it. I figured it's around the src-build and client build (there is a comment about it in the code) but I didn't find how to make it work.
Performance is not an issue because it's a demo I want to show my peers.#2023-07-1010:28Dustin Getzi don’t understand fully yet, please add me to your github repo#2023-07-1010:31Reut SharabaniI'll try and create one with a minimal example later. I can't add to my repo since it includes some private stuff.#2023-07-1010:34Reut Sharabani(I don't think it's related to hyperfiddle at this point, I'm posting here to leverage slack search which indexes these messages and can maybe help someone from the future with similar problems)#2023-07-1011:33Reut Sharabaniok so it turns out optimizations :none is not an option. You have to use optimizations :simple . The code even suggests that (setting optimizations in build-client to false, I hard coded the compiler option to :none)
This enables to requires through script tag without breaking anything.
@U09K620SG now it works and my diagrams are generated. Optimizations are not a concern for my demo.
Thank you so much for your help and patience!#2023-07-1011:33Reut Sharabanifor reference this is the error optimizations :none generates:
optimizations set to :none, can't optimize
{}
ExceptionInfo: optimizations set to :none, can't optimize
shadow.build.closure/optimize (closure.clj:1308)
shadow.build.closure/optimize (closure.clj:1301)
shadow.build.api/optimize (api.clj:273)
shadow.build.api/optimize (api.clj:267)
shadow.build/optimize (build.clj:525)
shadow.build/optimize (build.clj:517)
shadow.cljs.devtools.api/release* (api.clj:339)
shadow.cljs.devtools.api/release* (api.clj:324)
shadow.cljs.devtools.api/release/body-fn--19691--auto----19795 (api.clj:358)
shadow.cljs.devtools.api/release (api.clj:356)
shadow.cljs.devtools.api/release (api.clj:351)
build/build-client (build.clj:32)
build/build-client (build.clj:22)
build/uberjar (build.clj:49)
build/uberjar (build.clj:38)
clojure.lang.Var.invoke (Var.java:384)
clojure.run.exec/exec (exec.clj:89)
clojure.run.exec/exec (exec.clj:78)
clojure.run.exec/-main/fn--237 (exec.clj:228)
clojure.run.exec/-main (exec.clj:224)
clojure.run.exec/-main (exec.clj:192)
clojure.lang.Var.applyTo (Var.java:705)
clojure.core/apply (core.clj:667)
clojure.main/main-opt (main.clj:514)
clojure.main/main-opt (main.clj:510)
clojure.main/main (main.clj:664)
clojure.main/main (main.clj:616)
clojure.lang.Var.applyTo (Var.java:705)
clojure.main.main (main.java:40)
And the resulting jar doesn't load the a web page (missing manifest).#2023-07-1012:35Dustin GetzMy best guess is that shadow-api/release is intended to build a single .js file artifact and therefore an optimization strategy is required#2023-07-1012:35Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/16fff5da32b0a1f50d221cba56c872632ee10d2d/src-build/build.clj#L30C4-L30C22#2023-07-1012:36Dustin Getzi think you'd write shadow-api/compile for a dev build iirc#2023-07-1012:37Reut Sharabaniyes, I basically hard-coded an override of optimizations to :none when you directed me that the script tag inclusion doesn't work because of :advanced. This broke shadow's output.
So using :simple instead of :advanced (like the code you point to suggests) worked.#2023-07-1012:39Reut SharabaniSo when you want to include something that has lazy-loaded libraries internally, or that you can't npm-install for some other reason, you probably want to use :simple optimizations. Otherwise shadow messes with js in some way that breaks script tags (probably doesn't compile them alongside the npm_modules and whatever else).#2023-07-1015:39Dustin GetzIf you must, you can access the global by string, i.e. js.window["mermaid"] to bypass the name mangling#2023-07-1015:40Dustin Getz(def mermaid (aget js/window "mermaid")) ; something like this#2023-07-1015:40Reut SharabaniI'm focused on other things but I'll keep that in mind#2023-07-1015:40Reut Sharabaniit's a shadow issue anyway so I'm sure other devs just know how it works#2023-07-1012:37Reut Sharabaniyes, I basically hard-coded an override of optimizations to :none when you directed me that the script tag inclusion doesn't work because of :advanced. This broke shadow's output.
So using :simple instead of :advanced (like the code you point to suggests) worked.#2023-07-1004:04grounded_sageI am using the GPT api and electric seems to crash sometimes. I am assuming it is a timing out issue?#2023-07-1010:15Geoffrey GaillardCould you share more info to help us diagnose the issue?
What does the crash looks like?#2023-07-1017:56Dustin Getz@U05095F2K can you add me to your repo i want to check your entrypoint#2023-07-1006:28HendrikI have general question when things are executed in an e/defn . I have prepared this example
(e/defn my-fn [x y]
(println "executed once when my-fn is mounted into the dag")
(println "executed on mount and whenever x changes" x)
(println "executed on mount and whenever y changes" y)
(e/on-unmount #(println "executed once on unmount"))
(e/on-unmount #(println "executed once on unmount, too")))
Are these statements correct?
If so, is there an order in which things are executed. I read that everything is async.
E.g:
(e/defn my-fn [x]
(println "must run and finish first" x)
(println "must run after above has finished" x))
Is the second example true? If not, how can I enforce this?#2023-07-1009:48braai engineerYou could probably wrap them in a (do ...) statement but then both will run each time.#2023-07-1009:53HendrikElectric (do ...) does execute statements synchronous in order?
That would be fine for the second example because both statements rely on the same value#2023-07-1009:55braai engineerCould also move the println's out to a Clojure function:
(defn do-both [x y]
(println "executed on mount and whenever x changes" x)
(println "executed on mount and whenever y changes" y))
(e/defn my-fn [x y]
(do-both x y))
#2023-07-1010:13Geoffrey GaillardIn Electric Clojure (do …) executes statements in parallel.
I understand you want to sequence some effects. What’s your use case?#2023-07-1011:23braai engineerOh wow, I did not expect that. What about (e/server (do ...))?
What happens with a doseq?
This might explain why I had weird state related stuff happening with checkbox some weeks ago...
How do you force something to run in sequence that does not have symbolic dependency, m/sp?
e.g.
(do
(d/transact conn ...)
(d/q ... (d/db conn))
#2023-07-1012:09Hendrik@U2DART3HA I am trying to wrap three.js webgl library. I think that electrics RAII features are a perfect fit for cleaning up GPU resources.
Some example usecase:
;e.g render after resize. pseudo code
[width height] (stream-of-resize-events)
(.setSize renderer width height)
(.render renderer scene)
#2023-07-1012:18Geoffrey GaillardThree.js API is essentially imperative whereas Electric Clojure is functional and reactive. Today's short answer is to move your imperative code into clojure land, so it follows Clojure semantics.
;; Anonymous fn reruns on each !my-atom change
((fn [value] (prn "first") (prn "second")) (e/watch !my-atom))
#2023-07-1012:23Geoffrey Gaillard@U051SPP9Z you'd watch the datomic tx log and recompute (d/db conn) which in turn recomputes (d/q db). If you really need to perform d/q just after transact, then move the code to a Clojure function.
doseq behaves as in Clojure. for also. They evaluates sequentially. e/for evaluates in parallel.#2023-07-1012:35braai engineerUnderstand that e/for is parallel, but to confirm in (do (a) (b)), a and b will execute in parallel? I’m guessing in two separate threads?#2023-07-1012:39Geoffrey GaillardYes (a) and (b) will execute in parallel. But in the same thread (JavaScript is usu single threaded). If you want to run (a) or (b) on a different thread (JVM only) have a look at e/offload.#2023-07-1012:42Hendrikisn’t running in the same thread in JS Land synchronous? Or is the order which runs first not determined?#2023-07-1012:54Geoffrey GaillardThe order of which runs first after the initial run is not determined. Because Electric compiles you program to a graph and maintains it.
Let me clarify:
(def !my-atom (atom 0))
(e/def my-value (e/watch !my-atom))
(e/run (do (prn "No dependency")
(prn "One dependency:" my-value)
(prn "Again one dependency: " (inc my-value))))
This program will print immediately:
"No deppendency"
"One dependency 0"
"Again one dependency 1"
These lines will print in order because it’s the first run.
The program says up! So when you (swap! !my-atom inc) only what depends on !my-atom is recomputed:
"One dependency 1"
"Again one dependency 2"
The order of these two lines is not guaranteed.#2023-07-1013:06Hendrikthanks for the clarification#2023-07-1015:18Dustin GetzWhat does "parallel" mean? Electric is a massively concurrent DAG language, which means we automatically maximize concurrency at every point. (to the extent that we can without breaking Clojure compatibility)
(do (println x) (println y)) has ambiguous order, because x or y can be remote, in which case there may be latency when the expression "boots" (runs for the first time). y can be available first if x is remote.
In subsequent reactive changes to x or y, Electric will maximize concurrency and recompute only the expression whose arguments changed.
If both reactive values are available when the do expr boots, the println exprs will compute (run) in lexical (natural) order.
(do (println 1) (println 2)) will print 1 2 in that order (literals are never remote), and then never print again (literals never update).
---
(dom/div (do (dom/text x) (dom/text y))) will perform point writes in ambiguous order (like println), but with an additional rule: the DOM has a notion of order — an element's children are ordered — so we will ensure that the final result is <div>{$x}{$y}/> even if y is written first. Note the do is implicit, I phrased it explicitly for clarity.
On lexical ordering in the Electric DAG: a DAG node's list of edges are lexically ordered.
Electric provides an undocumented api e/hook to implement resource effects that have a notion of positional order and lifecycle, like dom elements. println and other side effects do not have such a notion of positional ordering and therefore do not opt into that API.
---
(let [] (dom/text x) (dom/text y)) – same as above because of implicit do. Thus similarly all other clojure forms that imply do such as when e/defn
(do (dom/div (dom/text x)) (dom/div (dom/text y))) will behave as per the above rules, but we must realize that div is in fact a macro, so the evaluation order has been mucked with to get declarative syntax (i.e., the div element must be first created so that the text element can then be attached). This is best understood by reading the source code of dom/element and dom/text which are about 10 lines total.
@U053XQP4S @U2DART3HA please confirm I have not made any errors, we also may need to tighten some vocabulary#2023-07-1015:29Dustin GetzSpecifically we need to clarify our usage of "parallel" vs "concurrent" i think#2023-07-1015:49HendrikDustin thank you for your detailed explanation. Now, I have a much better understand on how electric is working 🙂#2023-07-1113:11braai engineer@U09K620SG is it a good idea to use the same semantic "do" for a process with no guaranteed execution order? Maybe ado or pdo ? Just like Clojure has pmap (which is lazy tho).
This has tripped me up and will probably trip others up too.#2023-07-1113:30Dustin Getzyes, it is this way because it cannot reasonably be any other way, this was hard fought knowledge#2023-07-1113:33Dustin GetzTo be clear: electric's implementation of (do) is fully backwards compatible with clojure; if you copy paste pre-existing Clojure code into Electric (or call a pre-existing Clojure macro), the electrified expression will return the exact same result as it would in clojure#2023-07-1113:35Dustin Getzincluding running the effects in the exact same order, as demonstrated by (do (println 1) (println 2))#2023-07-1115:00leonoel> fully backwards compatible with clojure
almost. (do (assert false) (println 1)) doesn't print in clojure but prints in electric#2023-07-1116:56Dustin GetzRecalling now Leo's correction, let us consider:
(def !x (atom true))
(e/run (do (dom/text "a") (assert (e/watch !x) "b") (dom/text "c")))
(swap! !x not)
for clarity, e/run marks an electric reactive expression as if in a unit test
Should text element "c" be destroyed and removed from the DOM when !x toggles? What about "a"? What happens when !x toggles back?#2023-07-1008:51braai engineerIs (e/server (do-thing) nil) equivalent to (e/server (e/discard (do-thing)))?#2023-07-1010:01Geoffrey Gaillardyes#2023-07-1009:47braai engineerSo excited for incremental compilation ⚡#2023-07-1009:58braai engineerWhich cache does HYPERFIDDLE_ELECTRIC_SERVER_VERSION bust? Could server not set E-Tag on index.html and/or the JS it serves? On connect check version and if server & client don't match, force client reload?#2023-07-1010:11Geoffrey GaillardHYPERFIDDLE_ELECTRIC_SERVER_VERSION will trigger a client page refresh if it is set and doesn’t match HYPERFIDDLE_ELECTRIC_CLIENT_VERSION (hardcoded at build time)
About ETAGs.
Websocket client has auto-reconnect.
If a new server version is deployed (e.g. v1 to v2), the websocket client (v1) will be disconnected and will reconnect to sever (v2).
At this point there’s no HTTP request loading HTML or JS. Only the Websocket Upgrade. So no ETAG if I’m not mistaken.#2023-07-1111:25braai engineerIf HYPERFIDDLE_ELECTRIC_SERVER_VERSION is hardcoded at build time why does it need to be passed in when running uberjar? https://github.com/hyperfiddle/electric-starter-app#deployment
Anything that tries to start the JAR needs to know something about its version which does not make sense to me, even though it can't change the version. Makes it hard to deploy via Dokku in Procfile.#2023-07-1115:20Dustin Getzi believe this because today the dag is actually baked into the client, the server is general#2023-07-1115:27Dustin Getzperhaps the flag can be removed with minor upgrades to build.clj in the starter app#2023-07-1011:16braai engineerHow would the Datomic tx listener change to support separate Datomic databases per customer, but still stored in the same Datomic System and one app deployed?
I'd have to establish a separate conn per customer when they log in and hold onto it somewhere. I imagine I would have to move !db & !taker into a map per authenticated username / tenant ID?
Could I use the !present pattern but instead of usernames, put Datomic connections or DBs in there?
How do I create session-specific atoms? I would like to avoid watching a singleton atom with everyone's DB values that triggers a refresh for all customers any time anyone transacts anywhere.#2023-07-1012:30Dustin Getz(def !x (atom .)) ; global state
(let [!x (atom .)) ; local state#2023-07-1013:26braai engineerBy session-specific I mean auth-scoped so that a Datomic connection can be shared between two user windows, which I guess will mean having a global conns map keyed on user ID => conn and then subscribing/unsubscribing from txReportQueue.
I recall there being a limit, i.e. only a single txReportQueue per...connection? Or are Datomic connections relatively cheap? I imagine that each connection acts like own peer with own cache? So to share datom cache you'd probably want to share conn between Electric sessions with same tenant / user ID?#2023-07-1014:47Dustin GetzThese are expert level Datomic questions, i'd ask in their support channel#2023-07-1014:53mgxmI'ts possible to dynamic call and dispose an electric component?
• Something like that:
◦ 1) Client receives an string representing a component, ex: "ComponentA"
◦ 2) Client looks into registry for "ComponentA"
◦ 3) Client call new on that Component.
◦ 4) Client can dispose (unmount) this component in other parts of the code.
#2023-07-1015:35Dustin Getzmount/unmount are implicit, userland cannot explicitly dispose#2023-07-1015:35Dustin GetzShow me some pseudo-code for what business problem you are trying to accomplish#2023-07-1015:36Dustin Getzin https://electric.hyperfiddle.net/user.tutorial-lifecycle!Lifecycle note that the blinker is added/removed to the DOM but there is not an explicit disposal, you can also see in the console that we've hooked the mount/unmount lifecycle#2023-07-1016:08mgxmI'm thinking of way to integrate with https://golden-layout.com/.
• There are two ways to integrate.
◦ 1) Register components in their own way.
▪︎ the lib will mount/unmount the component in its own way.
◦ 2) Register two callbacks, bindComponent and unbindComponent
▪︎ here, it's up to you to mount/unmount the component on the dom
#2023-07-1016:13mgxmpseudo-code for the idea:
(GoldenLayout. dom/node
(fn bind
[container item]
(let [name (.-componentName item)
el (.-rootElement container)
resolved (get registry name)]
(dom/with el
(new resolved))))
(fn unbind
[container]
(let [el (.-rootElement container)]
(dispose el))))#2023-07-1017:21Dustin GetzElectric component inside GoldenLayout element?#2023-07-1017:34mgxmyes#2023-07-1017:38Dustin Getzthis might help https://github.com/hyperfiddle/fulcro-electric-template/blob/47e9340f7b9f69759a5f58831eb915b06b71e54d/src/main/hyperfiddle/electric/fulcro_dom_adapter.cljc#L48-L51#2023-07-1017:39Dustin Getzthis is a POC of a fulcro adapter, though in this configuration each instance of the with-electric bridge will get it's own websocket to the server (which works fine)#2023-07-1017:51mgxmgreat.. thanks#2023-07-1015:52JHi! Is there emacs users here (with clojure-lsp)? What is your hack to disable Unresolved namespace :
(ns app
#?(:clj [app.db :as db]))
(e/server (db/foo ...)) ;; Unresolved namespace `db
`#2023-07-1111:25braai engineerIf HYPERFIDDLE_ELECTRIC_SERVER_VERSION is hardcoded at build time why does it need to be passed in when running uberjar? https://github.com/hyperfiddle/electric-starter-app#deployment
Anything that tries to start the JAR needs to know something about its version which does not make sense to me, even though it can't change the version. Makes it hard to deploy via Dokku in Procfile.#2023-07-1112:11braai engineerIs the following nginx.config fine for Electric apps to map 8080 to port 80 + websockets? Or should I just host Electric app directly on port 80?
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass "";
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
Most nginx examples have separate websocket endpoints e.g. at /ws.#2023-07-1115:18Dustin Getzi dont know#2023-07-1118:49Vincentmy nginx configuration works out of the box with websockets over port 80 with proxypass to 8080 for the electric app.#2023-07-1118:50Vincentif you want custom ports, lancelot be with you 😅#2023-07-1118:50Vincent(one of the knights of the round table :p)#2023-07-1118:52Vincentserver {
server_name your_domain_dot_co;
location / {
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass ;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
}
and also a listen 80; in there. and you should be Good2GO#2023-07-1118:55Dustin Getzbro that whitespace#2023-07-1119:06Vincentjust copying and pasting from the ssh prompt lol, it's tabs i swear#2023-07-1119:07Vincentstyle and whitespace updates are for employees 50-1000 lol, 1 and 2 gotta keep moving laughcry#2023-07-1118:40Dustin GetzStarter app readme now documents error reporting and logging. If your error reporting is not working, something is wrong (perhaps you have a very old fork of the starter app), please validate your configuration! https://github.com/hyperfiddle/electric-starter-app#2023-07-1118:54telekidjust started an experimental branch integrating electric into an existing datalevin-based app. After about a day of work, I managed to get cljs + electric integrated into my clojure app, built a small (probably not optimal) reactive datalevin listener, and have updates streaming to my UI. This stuff is complete magic.#2023-07-1118:54Dustin GetzNice, @U050CJFRU also a datalevin user#2023-07-1118:54Dustin GetzPost the listener#2023-07-1118:56denik@UABU2MMNW for reference here’s the listener I use https://github.com/lumberdev/tesserae/blob/04d168ddaf7d5cc8ee56201038a34a48df11d4d3/src/tesserae/ui/electric_util.cljc#L115-L128#2023-07-1118:56telekid#?(:clj
(defn latest-db>
"return flow of latest datalevin transaction"
[!datalevindb]
;; TODO:
;; This is cribbed and modified from here:
;;
;; The xtdb version sets the initial value to the last-known tx
(->> (m/observe (fn [!]
(let [listener (dlc/listen! !datalevindb !)]
#(.close listener))))
(m/reductions {} nil) ; initial value is the latest known tx, possibly nil
(m/relieve {})
(m/latest (fn [_]
(dlc/db !datalevindb))))))
#2023-07-1118:56denikused here https://github.com/lumberdev/tesserae/blob/04d168ddaf7d5cc8ee56201038a34a48df11d4d3/src/tesserae/ui/views.cljc#L286#2023-07-1118:56telekidnice! I'll read through that#2023-07-1118:57telekidjust starting to dive into missionary#2023-07-1118:57denikbut yeah the combo of datalevin + electric is incredible#2023-07-1118:59telekidit's wild.#2023-07-1119:07JMaybe a naive question but why just use e/watch on the datalevin conn?#2023-07-1119:18denikthis may be okay. I had an issue previously where the server thread would deadlock. further investigation identified a memory leak in datalevin which has since been resolved. in the meantime @U053XQP4S had suggested to use this listener.#2023-07-1120:51telekiddoes e/dom expose something equivalent to react fragments, allowing you to return a collection of elements that expand into the containing dom node?#2023-07-1120:52Dustin GetzShow me your use case, many use cases of react fragments are solved by effectful rendering style#2023-07-1120:53Dustin GetzYou can also return (e/fn [] (dom/div) (dom/div)) as a value, pass around, invoke somewhere else#2023-07-1120:53Dustin GetzNote it contains two divs without needing a fragment#2023-07-1120:58telekidAh! That will do.#2023-07-1208:10braai engineeryeah I am now sold on the effectful style. Much more flexible for complex views#2023-07-1210:09Dustin Getzindeed, it is quite shocking isn’t it#2023-07-1208:59HendrikI try to create a ref similar to dom/node . I tried to start with something simple, but I already get some weird errors:
(e/def some-ref)
(defmacro with-ref [ref & body]
`(binding [some-ref ~ref]
~@body))
(e/defn bar []
(println "mount" some-ref))
(e/defn Some []
(e/client
(binding [some-ref 24]
(bar.))
(with-ref 42 (bar.))
))
The manual (binding [some-ref 24] (bar.)) works. However, if I replace it with the with-ref macro, then I get (error in the answer). Can anybody explain what is happening here and what is causing this?#2023-07-1209:01HendrikUnbound var `app.todo-list/some-ref`
in ( clojure.core/println mount <exception> ) in app/todo_list.cljc
in reactive (defn bar [] ...) in app/todo_list.cljc line 46
in ( app.todo-list/with-ref 42 <exception> ) in app/todo_list.cljc
in reactive (defn Some [] ...) in app/todo_list.cljc line 49
in (try ...)
Error: Unbound var `app.todo-list/some-ref`
at Object.hyperfiddle$electric$impl$runtime$error [as error] (runtime.cljc:66:3)
at eval (runtime.cljc:394:28)
at eval (Latest.cljs:34:42)
at missionary$impl$Latest$transfer (Latest.cljs:44:45)
at Object.eval [as cljs$core$IDeref$_deref$arity$1] (Latest.cljs:10:17)
at Object.cljs$core$_deref [as _deref] (core.cljs:688:12)
at Object.cljs$core$deref [as deref] (core.cljs:1477:4)
at eval (Latest.cljs:28:20)
at missionary$impl$Latest$transfer (Latest.cljs:44:45)
at Object.eval [as cljs$core$IDeref$_deref$arity$1] (Latest.cljs:10:17)#2023-07-1217:01Dustin GetzThe code looks correct to me, could it be the same issue as your subsequent thread?#2023-07-1217:08Dustin GetzThis works for me in the starter app repo:#2023-07-1217:08Dustin Getz#2023-07-1217:08Dustin Getz#2023-07-1217:10Dustin GetzPossible gotchas are:
• e/def duplication/ambiguous site
• possibly hot code reloading issues related to forgetting the :require-macros the first time, causing the cljs build to end up in a bad state? This happens infrequently enough that there may be latent issues not yet detected at the electric/cljs/shadow touch points. We do have a custom hot code reloading strategy that forces the server to reload every .cljc file that shadow rebuilds. (That is necessary to keep your client/server code versions in sync)#2023-07-1218:45HendrikThanks for your response. I think that it was a hot code reloading and forgetting the :require-macros issue. Now it works :)#2023-07-1219:48Dustin GetzOk, yeah I think i saw the exact same issue when testing this (I also forgot the require-macros), which means we have repro steps for the next time we work on this#2023-07-1210:12braai engineerWill Electric work with Quarkus / GraalVM? (`native-image`)#2023-07-1213:55leonoelafter incremental ~computation~ compilation is landed there should not be any technical blocker for AOT compilation anymore#2023-07-1300:27Vincentincremental compilation? salivates uncontrollably #2023-08-3117:45grounded_sageThis is definitely desired#2023-07-1213:31telekidFWIW after some initial grumbling, I'm not missing hiccup syntax as much as I thought I would#2023-07-1214:07teodorluWhat’s the advantage of hiccup over the current syntax?
Are there specific use cases that you feel are lacking now that would improve with support for hiccup?#2023-07-1214:28telekid> What’s the advantage of hiccup over the current syntax?
I'm not convinced there are any#2023-07-1214:29telekidbut will pay attention as I go forward#2023-07-1214:30telekidI think it's hard to understand why hiccup isn't an improvement until you start to become comfortable with the effectful rendering style#2023-07-1214:35jjttjjAgreed. I think it's a prime example of people irrationally preferring something that's familiar to them. Hiccup is for sure a great way to represent flat html files, but it's a poor fit imo for dynamic web apps. Personally I almost prefer the dom-elements-as-functions anyway. It's natural to me that a div would be a function that mutates the dom rather than a data structure created by a function that is handed off to some reconciliation engine.
The overhead is pretty big (in terms of performance and/or complexity you take on) to get hiccup syntax into cljs dom libraries and at the end of the day you basically get to just use square brackets and keywords instead of symbols and parens.
I really like the electric dom syntax. I kind of even like that it's a little more verbose, where in electric you must explicitly wrap props and text for example, because it makes sense to me that these should be explicit, since that is like, the granularity where things can change in your dom tree.#2023-07-1215:02Dustin GetzYeah, abstractionist persona falls in love with the precision, frontend dev persona (high end frontends are markup heavy, as of 2023 at least) hates the boilerplate#2023-07-1215:04Dustin GetzThe syntax I have in mind is this:
(dom/h1 :text "hello world")
(dom/h1 ... {:text "hello world"})
critically it separates static props (that are always present) from dynamic props (which may be present), e.g (dom/h1 ... (if x (assoc {} :text "hello world")))#2023-07-1215:05Dustin GetzBut I have not thought about it in a while, it needs to be checked carefully for ambiguous edge cases#2023-07-1215:05Dustin Getz... is "spread" operator, or perhaps use ߧ-07-1215:08Dustin Getzplus there is reasonable sugar that can be added on top, something like :div.class1.class2#id can probably be hacked in somehow for the markup persona who writes a lot of static annotations#2023-07-1408:30braai engineerWould be nice to have some sugar because typing out props ... :class is everywhere. Instead of:
(dom/div
(dom/props {:class (string/join " " [(if active? "active) "other-class"])})
(dom/text "hi" name))
would be nice if it could be:
(div.other-class {:class [(if active? :active)]}
"hi" name) ;; can dom/text be assumed? probably edge cases.#2023-07-1408:31braai engineer@U09K620SG how about vectors instead of maps? [:text "hello " name] ?#2023-07-1214:34Reut Sharabaniis there an example about saving state per client?#2023-07-1214:35Reut Sharabani(like sessions...)#2023-07-1214:39s-olserver side session state and client UI state work the same way, just in e/server and e/client#2023-07-1214:39s-olyou just need an atom and e/watch:#2023-07-1214:39s-ol(let [!val (atom initial-val)
val (e/watch !val)]
...)
#2023-07-1214:40Reut SharabaniI needed to know where I can generate an "id" since the trivial examples it's transparent (there is no request).
The example I attached shows where the websocket request data is stored if I understand corretly.#2023-07-1214:40s-oluse swap! etc to update !val, val streams changes#2023-07-1214:41Reut SharabaniI didn't try it yet but I see it has cookies and stuff so I expect it will be stable. Why do you think it won't be?#2023-07-1214:41s-olif you need it to mediate between clients#2023-07-1214:42Reut Sharabanihow can a client know what state belongs to them? You have to diffentiate them somehow. Maybe there is a pattern I'm missing here. An example would be helpful.#2023-07-1214:43s-olif the state is not persistant across reloads you don't need to, the electric entrypoint is "instantiated" server-side once for each websocket/client#2023-07-1214:44s-olso any local state declared is per-client already, no need for an id#2023-07-1214:44s-olif you want to share state between clients you do that via global atoms, but that's optional#2023-07-1214:45Reut Sharabaniyou're saying this code belongs in the client?
(let [!val (atom initial-val)
val (e/watch !val)]
...)
#2023-07-1214:45Reut SharabaniI thought you mean to put this in the server#2023-07-1214:46Reut Sharabanibut if you have a good canonical example I think it's valuable to attach it here so it's indexed by slack#2023-07-1214:49s-ol@U7KDU667Q if you put this on the server, each client will have its own !val#2023-07-1214:50s-olhttps://electric.hyperfiddle.net/user.tutorial-7guis-1-counter!Counter#2023-07-1214:51s-olthis is an example: the counter state is per-client (always starts at 0, separate if you open two tabs)#2023-07-1214:53Reut SharabaniI understand the "1-counter" example but I want a state per client on the server.
I don't understand this statement:
if you put this on the server, each client will have its own !val (edited)
From my experience it's shared for all clients it's not per-client.
Maybe I don't understand how my code works 😄
This var (`mmd`) holds a graph description:
#?(:clj (defonce !mmd (atom "")))
(e/def mmd (e/server (e/watch !mmd)))
When I use it in e/client it's shared across clients. So I'm not sure what you mean.#2023-07-1215:08s-olthis is an e/def, not the sample i shared#2023-07-1215:09s-oltry this:
(ns user.tutorial-7guis-1-counter
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
(e/defn Counter []
(let [!state (atom 0)
state (e/watch !state)]
(e/client
(dom/p (dom/text state))
(ui/button (e/fn [] (e/server (swap! !state inc) nil))
(dom/text "Count")))))#2023-07-1215:09s-olthis is per client state on the server#2023-07-1215:11s-olthink about it this way: adding or removing e/server or e/client should never change the behaviour of the program. (In practice, there is a CLJ vs CLJS difference, so many things will actually break)#2023-07-1215:17Dustin Getz• (def !x (atom .)) global state, shared. (Clojure semantics)
• (let [!x (atom .)] ...) local state, not shared. !x extent is bounded by the electric function's extent which is bounded by the electric program's extent which is tied to the websocket connection extent
• (e/def !x (e/server (atom .))) local because e/def extent is tied to the program extent which is tied to the websocket connection extent. i think we are reconsidering this, it was an oversight. It might not change#2023-07-1215:27Dustin GetzAnother way to see it is that Electric bindings have an object lifecycle = dynamic extent. Clojure globals have indefinite extent = the entity continues to exist as long as the possibility of reference remains (consider garbage collection). See https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html#2023-07-1215:35HendrikIs it possible to use a (e/def db) in a callback?
I added this to the starter app:
(ui/button (e/fn [] (println (todo-count db))) (dom/text "click"))
After clicking the button I get Unbound var app.todo-list/db`` . Is it possible to make (e/def db) available at callback time?#2023-07-1216:55Dustin GetzIt is possible and works, with a gotcha (that we're planning to fix):
Today, (e/def x 1) creates a binding on both client and server, there are two bindings with the same name. Since it is ambiguous, you must be explicit from which site you resolve it from. This is in contrast with let bindings which are defined unambiguously at a single site and therefore the compiler can infer which site has the binding.
(e/def x 1) ; ambiguous, binding x exists at both places
(e/server
(let [y 2] ; unambiguous, binding y located at server
(e/client
(println y) ; y inferred to be on server
(println x) ; x is ambiguous so compiler assumes local access, here client
(println (e/server x)) ; explicitly access x on server
)))
the middle println will produce unbound var runtime error#2023-07-1216:56Dustin GetzSo in your case, I expect that db is only bound on the server and therefore we need to explicitly access it on the server#2023-07-1216:56Dustin GetzResolving the ambiguity is WIP#2023-07-1217:19Dustin GetzI opened a public ticket to track https://github.com/hyperfiddle/electric/issues/46#2023-07-1219:01HendrikThanks again for the explanation. I did not see that db was bound server side. Now I got it to work. I chose db and the starter app to have a simple reproducible example. My usecase is still getting three.js opengl wrapped into electric. A rerender must be called explictly. So my idea is to have is_dirty state, which is created by the renderer and is passed to members of the scenegraph. Similar to e/dom where children and props can rely on something up the tree is setting it to the right value.#2023-07-1219:45Dustin GetzIs the scene graph mutable (threejs data structure) or is it clojure data? Clojure equality semantics are the perfect cachekey for establishing such a dependency in the DAG to cause render to happen#2023-07-1219:45Dustin Getza dirty flag (i.e. a counter) is fine also#2023-07-1219:47Dustin GetzIf you add me to your repo with your attempt (even messy scratch stuff) I can take a look#2023-07-1223:42braai engineerIf you e/def dynamically, how do you undef the var? (e/def x nil)?#2023-07-1305:47HendrikIt is a mutable threejs data structure. Therefore the dirty flag. In theory I could create a immutable data structure by copy on write. That would require a recreation of the scene graph up to the root node. Maybe that is efficient enough. At least it requires no dirty flag. For now I have no repository, yet, just some code snippets. I’ll create one at the weekend.#2023-07-1217:50grounded_sageFound a subtle error that could have a friendlier error message, when there is a space starting at the class list:
(dom/props {:class " my-class")
Error:
core.cljs:3953 Cannot read properties of null (reading 'cljs$core$IFn$_invoke$arity$0')
in reactive (fn [] ...)
in reactive (fn ClassList [node classes] ...)
in reactive (fn [] ...)
in reactive (defn WebsiteGreeting [] ...) in app/todo_list.cljc line 485
in reactive (defn App [] ...) in app/todo_list.cljc line 503
in (try ...)
TypeError: Cannot read properties of null (reading 'cljs$core$IFn$_invoke$arity$0')
at eval (Relieve.cljs:34:27)
at Object.missionary$impl$Relieve$ready [as ready] (Relieve.cljs:35:44)
at G__40067 (Relieve.cljs:42:27)
at Object.missionary$impl$Observe$run [as run] (Observe.cljs:40:13)
at eval (core.cljc:564:27)
at eval (Relieve.cljs:42:8)
at Object.missionary$impl$Relieve$run [as run] (Relieve.cljs:42:7)
at eval (core.cljc:659:34)
at eval (runtime.cljc:601:13)
at eval (Continuous.cljs:144:8)#2023-07-1217:51Dustin GetzWhat electric version? I think that might have been fixed#2023-07-1221:50grounded_sageOh it might. I haven’t checked on the version.#2023-07-1221:52Dustin Getzit might be fixed only on master actually, if so better just to wait #2023-07-1223:39braai engineerIs this supposed to work?
(e/defn TextInput [value OnChange]
(ui/input value OnChange
(dom/props {:placeholder "some placeholder..."})))
(e/defn App []
(let [!text (atom ""), text (e/watch !text)]
(TextInput. text (e/fn [v] (reset! !text v)))))
ui/input works if I put it directly in App but does not seem to work if I move it to a subcomponent.#2023-07-1300:17Dustin Getzi don’t see a problem here, try it in a fresh starter app to confirm#2023-07-1300:18Dustin Getz(the stack trace does not match the code, so it seems that more is in play than this snippet)#2023-07-1316:17braai engineerResolved by bumping to latest Electric version.#2023-07-1313:04Reut SharabaniI showed electric at work today and I described it as a way to write "fullstack functions"#2023-07-1313:10Dustin Getzhow did people respond#2023-07-1314:07Reut Sharabanithey liked it. Took them a while to understand because it was two backend-only devs but I could "hear" it when it clicked for them#2023-07-1316:17grounded_sagePeople out there writing one function serverless functions. We over here writing the whole application as a function. #2023-07-1313:49braai engineerException on Uberjar build: "Method code too large"
15:46:55,640 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@151bf776 - Registering current configuration as safe fallback point
Execution error (IndexOutOfBoundsException) at clojure.asm.MethodWriter/computeMethodInfoSize (MethodWriter.java:2061).
Method code too large!
Full report at:
/var/folders/3y/0kx0ddld5m116s7p_62h1ynr0000gp/T/clojure-11423849822174727895.edn
Execution error (ExceptionInfo) at clojure.tools.build.tasks.compile-clj/compile-clj (compile_clj.clj:112).
Clojure compilation failed, working dir preserved: /var/folders/3y/0kx0ddld5m116s7p_62h1ynr0000gp/T/compile-clj11185829201170148305
Full report at:
/var/folders/3y/0kx0ddld5m116s7p_62h1ynr0000gp/T/clojure-12005292496436482601.edn
#2023-07-1314:07braai engineer$ cat /var/folders/3y/0kx0ddld5m116s7p_62h1ynr0000gp/T/clojure-1088563969155590368.edn
{:clojure.main/message
"Execution error (IndexOutOfBoundsException) at clojure.asm.MethodWriter/computeMethodInfoSize (MethodWriter.java:2061).\nMethod code too large!\n",
:clojure.main/triage
{:clojure.error/class java.lang.IndexOutOfBoundsException,
:clojure.error/line 2061,
:clojure.error/cause "Method code too large!",
:clojure.error/symbol clojure.asm.MethodWriter/computeMethodInfoSize,
:clojure.error/source "MethodWriter.java",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type clojure.lang.Compiler$CompilerException,
:message "Syntax error macroexpanding at (prod.clj:1:1).",
:data
{:clojure.error/phase :execution,
:clojure.error/line 1,
:clojure.error/column 1,
:clojure.error/source "prod.clj"},
:at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3719]}
{:type java.lang.IndexOutOfBoundsException,
:message "Method code too large!",
:at
[clojure.asm.MethodWriter
computeMethodInfoSize
"MethodWriter.java"
2061]}],
:trace
[[clojure.asm.MethodWriter
computeMethodInfoSize
"MethodWriter.java"
2061]
[clojure.asm.ClassWriter toByteArray "ClassWriter.java" 457]
[clojure.lang.Compiler compile "Compiler.java" 7909]
[clojure.lang.RT compile "RT.java" 411]
[clojure.lang.RT load "RT.java" 457]
[clojure.lang.RT load "RT.java" 424]
...#2023-07-1314:07braai engineerTrying to debug. When I eval my app namespace in REPL, I get this:
ERROR in () (:)
expected: (hyperfiddle.rcf/= (analyze {} (quote (try 1 (catch Exception e 2) (finally 3)))) [(ir/pub (ir/literal nil) (ir/bind 1 1 (ir/pub (ir/apply (assoc (ir/global :hyperfiddle.electric.impl.runtime/latest-first) :hyperfiddle.electric.debug/meta {}) (ir/apply (assoc (ir/global :hyperfiddle.electric.impl.runtime/bind) :hyperfiddle.electric.debug/meta {}) (assoc (ir/global :hyperfiddle.electric.impl.runtime/recover) :hyperfiddle.electric.debug/meta {}) (ir/apply (assoc (ir/global :clojure.core/some-fn) :hyperfiddle.electric.debug/meta {}) (ir/apply (assoc (ir/global :hyperfiddle.electric.impl.runtime/clause) :hyperfiddle.electric.debug/meta {}) (ir/apply (assoc (ir/global :clojure.core/partial) :hyperfiddle.electric.debug/meta {}) (ir/inject 0) (assoc (ir/constant (ir/pub (ir/apply (assoc (ir/global :hyperfiddle.electric.debug/unwrap) :hyperfiddle.electric.debug/meta {}) (assoc (ir/node 0) :hyperfiddle.electric.debug/name (quote hyperfiddle.electric.impl.compiler/exception) :hyperfiddle.electric.debug/scope :dynamic)) (ir/apply (ir/literal {}) (ir/sub 1) (ir/pub (assoc (ir/node 0) :hyperfiddle.electric.debug/name (quote hyperfiddle.electric.impl.compiler/exception) :hyperfiddle.electric.debug/scope :dynamic) (ir/apply (ir/literal {}) (ir/sub 1) (ir/bind 1 1 (ir/literal 2))))))) :hyperfiddle.electric.debug/type :catch :hyperfiddle.electric.debug/args (quote [Exception e]))) (assoc (ir/global :java.lang.Exception) :hyperfiddle.electric.debug/meta {}))) (assoc (ir/constant (ir/literal 1)) :hyperfiddle.electric.debug/type :try :hyperfiddle.electric.debug/meta nil)) (assoc (ir/constant (ir/literal 3)) :hyperfiddle.electric.debug/type :finally)) (ir/apply (ir/literal {}) (ir/sub 1) (ir/pub (ir/literal 0) (ir/apply (ir/literal {}) (ir/sub 1) (ir/bind 2 1 (ir/variable (ir/sub 2))))))))) (ir/do [] (ir/do [(ir/target []) (ir/target []) (ir/target []) ir/source] ir/nop))])
actual: clojure.lang.ExceptionInfo: Unable to resolve symbol: hyperfiddle.electric/trace
{:in [(clojure.core/binding [hyperfiddle.electric/trace hyperfiddle.electric.impl.compiler/exception] 2)]}
at hyperfiddle.electric.impl.compiler$analyze_form.invokeStatic (compiler.clj:643)
hyperfiddle.electric.impl.compiler$analyze_form.invoke (compiler.clj:628)
dunno if it could be related?#2023-07-1314:12braai engineerFixed by bumping Electric to latest starter app version: v2-alpha-349-ge9996713#2023-07-1316:01Dustin GetzWe have never seen this#2023-07-1317:18markaddlemanJVM method size is limited to 64k, I think. The way clojure compiles data structure literals, large data structures can exceed this limit. Easiest work around is to split the function into smaller functions#2023-07-1408:26braai engineerIt's possible there was some weird cache thing in target/classes and by changing deps it cleared cache 🤷 ? Not going to investigate further.#2023-08-1108:41braai engineerUsing v349 getting this Method code too large error again on a project I haven't touched in a while when building uberjar on host machine (not in Docker). I'm guessing it's something Java / deps related and not about Electric, so gonna try clear deps cache and see if it helps.
{:clojure.main/message
"Execution error (IndexOutOfBoundsException) at clojure.asm.MethodWriter/computeMethodInfoSize (MethodWriter.java:2061).\nMethod code too large!\n",
:clojure.main/triage
{:clojure.error/class java.lang.IndexOutOfBoundsException,
:clojure.error/line 2061,
:clojure.error/cause "Method code too large!",
:clojure.error/symbol clojure.asm.MethodWriter/computeMethodInfoSize,
:clojure.error/source "MethodWriter.java",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type clojure.lang.Compiler$CompilerException,
:message "Syntax error macroexpanding at (prod.clj:1:1).",
:data
{:clojure.error/phase :execution,
:clojure.error/line 1,
:clojure.error/column 1,
:clojure.error/source "prod.clj"},
:at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3719]}
{:type java.lang.IndexOutOfBoundsException,
:message "Method code too large!",
:at
[clojure.asm.MethodWriter
computeMethodInfoSize
"MethodWriter.java"
2061]}],
:trace
[[clojure.asm.MethodWriter
computeMethodInfoSize
"MethodWriter.java"
2061]
[clojure.asm.ClassWriter toByteArray "ClassWriter.java" 457]
[clojure.lang.Compiler compile "Compiler.java" 7909]
[clojure.lang.RT compile "RT.java" 411]
[clojure.lang.RT load "RT.java" 457]
[clojure.lang.RT load "RT.java" 424]
[clojure.core$load$fn__6908 invoke "core.clj" 6161]
[clojure.core$load invokeStatic "core.clj" 6160]
[clojure.core$load doInvoke "core.clj" 6144]
[clojure.lang.RestFn invoke "RestFn.java" 408]
[clojure.core$load_one invokeStatic "core.clj" 5933]
[clojure.core$load_one invoke "core.clj" 5928]
[clojure.core$load_lib$fn__6850 invoke "core.clj" 5975]
[clojure.core$load_lib invokeStatic "core.clj" 5974]
[clojure.core$load_lib doInvoke "core.clj" 5953]
[clojure.lang.RestFn applyTo "RestFn.java" 142]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$load_libs invokeStatic "core.clj" 6016]
[clojure.core$load_libs doInvoke "core.clj" 6000]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$require invokeStatic "core.clj" 6038]
[clojure.core$require doInvoke "core.clj" 6038]
[clojure.lang.RestFn invoke "RestFn.java" 619]
[prod$loading__6789__auto____140 invoke "prod.clj" 1]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3714]
[clojure.lang.Compiler compile1 "Compiler.java" 7748]
[clojure.lang.Compiler compile1 "Compiler.java" 7738]
[clojure.lang.Compiler compile "Compiler.java" 7815]
[clojure.lang.RT compile "RT.java" 411]
[clojure.lang.RT load "RT.java" 457]
[clojure.lang.RT load "RT.java" 424]
[clojure.core$load$fn__6908 invoke "core.clj" 6161]
[clojure.core$load invokeStatic "core.clj" 6160]
[clojure.core$load doInvoke "core.clj" 6144]
[clojure.lang.RestFn invoke "RestFn.java" 408]
[clojure.core$load_one invokeStatic "core.clj" 5933]
[clojure.core$compile$fn__6913 invoke "core.clj" 6171]
[clojure.core$compile invokeStatic "core.clj" 6171]
[clojure.core$compile invoke "core.clj" 6163]
[user$eval136$fn__137 invoke "compile.clj" 5]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
[clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[user$eval136 invokeStatic "compile.clj" 1]
[user$eval136 invoke "compile.clj" 1]
[clojure.lang.Compiler eval "Compiler.java" 7194]
[clojure.lang.Compiler load "Compiler.java" 7653]
[clojure.lang.Compiler loadFile "Compiler.java" 7591]
[clojure.main$load_script invokeStatic "main.clj" 475]
[clojure.main$script_opt invokeStatic "main.clj" 535]
[clojure.main$script_opt invoke "main.clj" 530]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]],
:cause "Method code too large!",
:phase :execution}}#2023-08-1108:46braai engineerDifferent error happens when building in a Docker:
#26 39.28 -> Closure - Optimizing ...
#26 42.93 Optimizing CLJS Constants took 187ms
#26 159.0 Execution error (StackOverflowError) at com.google.javascript.jscomp.CodeGenerator/add (CodeGenerator.java:201).
#26 159.0 null
#26 159.0
#26 159.0 Full report at:
#26 159.0 /tmp/clojure-11715170771542413787.edn
------
Dockerfile:24
--------------------
22 | ARG REBUILD=unknown
23 | ARG VERSION
24 | >>> RUN clojure -X:build uberjar :jar-name "opentax.jar" :version '"'$VERSION'"' :optimizations :advanced :verbose true
25 |
26 | FROM amazoncorretto:11 AS app
--------------------
ERROR: failed to solve: process "/bin/sh -c clojure -X:build uberjar :jar-name \"opentax.jar\" :version '\"'$VERSION'\"' :optimizations :advanced :verbose true" did not complete successfully: exit code: 1
#2023-08-1110:09braai engineerPushed to Dokku and it seems to build 🤷 . So must be deps state issue#2023-08-1110:21Dustin Getzi am traveling next 4 days but something to check is the heap size JVM flags, we customize this in the starter app#2023-08-1110:22Dustin Getzi think this probably is related to electric, we have macroexpansion size problems and compile time memory problems (incremental compilation is the next step)#2023-07-1317:08s-olI'm having issues with core.match:
> An error occurred while generating code for the form.
> ExceptionInfo: failed compiling constant: interface clojure.lang.IPersistentVector; java.lang.Class is not a valid ClojureScript constant.
#2023-07-1317:08s-olin e/server context:
(match path
[:link :tink]
"test"
:else
(new NotFound))))))))))#2023-07-1317:09s-olnamespace:
(ns seshat.core
....
#?(:clj (:require [clojure.core.match :refer [match]]
[seshat.state :refer [bindings]]
[datomic.api :as d])
:cljs (:require [cljs.core.match :refer-macros [match]])))#2023-07-1317:10s-olif ti comment out the first pattern it works. Seems like as soon as I introduce a vector pattern it stops working#2023-07-1317:10s-olempty pattern also breaks#2023-07-1317:11s-olotoh map patterns don't seem to upset it#2023-07-1317:11Dustin GetzI'm not surprised, just wrap it in a clojure lambda (inline or extract a fn) as a workaround#2023-07-1317:11Dustin GetzWe'd have to take a look at what it is doing, perhaps we are missing some feature, or perhaps it is doing Bad Stuff#2023-07-1317:11s-olbut if I want to call e/fn's inside?#2023-07-1317:11Dustin Getzoh, hmm#2023-07-1317:11s-olthis is supposed to be my router#2023-07-1317:12Dustin GetzSend me what you have so I can grok the specifics#2023-07-1317:13s-olmore complete / real world:
(let [[path query hash] history/location]
(e/server
(match path
[:link :tink & rest]
(new tink/LinkAccount rest query)
[:account type id]
(new account/View (e/client (keyword (name type) id)) hash)
:else
(new NotFound))))))))))#2023-07-1317:14Vincentnewb question: is new x the same as (x.) ?#2023-07-1317:15Vincentok cool. hmmm. make sure id is not a different value than you expect.#2023-07-1317:15Vincentmight need a different alias depending on where nested#2023-07-1317:15Vincent(my newb guess!)#2023-07-1317:15Dustin Getz@U05AL1ZH8TW might be worth trying one of the other match libs#2023-07-1317:15Dustin GetzI will need to remember how core.match works it has been a long time#2023-07-1317:16s-ol@U09K620SG for your reference, here's the macroexpand of a simpler case that also breaks:
(macroexpand '(match 123
[:a :b] "a b"
:else "else"))
=>
(let*
[ocr-56553 123]
(try
(clojure.core/cond
(clojure.core/and (clojure.core/vector? ocr-56553) (clojure.core/== (clojure.core/count ocr-56553) 2))
(try
(clojure.core/let
[ocr-56553_0__56555 (clojure.core/nth ocr-56553 0)]
(clojure.core/cond
(clojure.core/= ocr-56553_0__56555 :a)
(try
(clojure.core/let
[ocr-56553_1__56556 (clojure.core/nth ocr-56553 1)]
(clojure.core/cond (clojure.core/= ocr-56553_1__56556 :b) "a b" :else (throw clojure.core.match/backtrack)))
(catch
Exception
e__38244__auto__
(if
(clojure.core/identical? e__38244__auto__ clojure.core.match/backtrack)
(do (throw clojure.core.match/backtrack))
(throw e__38244__auto__))))
:else
(throw clojure.core.match/backtrack)))
(catch
Exception
e__38244__auto__
(if
(clojure.core/identical? e__38244__auto__ clojure.core.match/backtrack)
(do (throw clojure.core.match/backtrack))
(throw e__38244__auto__))))
:else
(throw clojure.core.match/backtrack))
(catch
Exception
e__38244__auto__
(if (clojure.core/identical? e__38244__auto__ clojure.core.match/backtrack) (do "else") (throw e__38244__auto__)))))#2023-07-1317:16Dustin Getzthanks#2023-07-1317:17Dustin Getzexceptions as control flow + electric smells like trouble, that hits us right in a problem area of interop#2023-07-1317:17s-olstack trace for that one:
[:dev] Build failure:
------ ERROR -------------------------------------------------------------------
File: /home/s-ol/Documents/other/seshat/src/user.cljs:7:1
--------------------------------------------------------------------------------
4 | hyperfiddle.electric
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
-------^------------------------------------------------------------------------
An error occurred while generating code for the form.
ExceptionInfo: failed compiling constant: interface clojure.lang.IPersistentVector; java.lang.Class is not a valid ClojureScript constant.
cljs.compiler/fn--4087 (compiler.cljc:306)
cljs.compiler/fn--4087 (compiler.cljc:303)
clojure.lang.MultiFn.invoke (MultiFn.java:229)
cljs.compiler/emit-constant-no-meta (compiler.cljc:285)
cljs.compiler/emit-constant-no-meta (compiler.cljc:277)
cljs.compiler/emit-constant (compiler.cljc:301)
cljs.compiler/emit-constant (compiler.cljc:297)
cljs.compiler/emit-constants-comma-sep/fn--4122/fn--4123 (compiler.cljc:417)
clojure.core/map-indexed/mapi--8638/fn--8639 (core.clj:7378)
#2023-07-1317:18s-olI was surprised to see how much stuff that little expression generates!#2023-07-1317:18Dustin Getzit's optimized (i think? in theory?)#2023-07-1317:19Dustin Getzis that in a e/server or an e/client?#2023-07-1317:19s-ole/server#2023-07-1317:19s-ol@U09K620SG any recommendation for core.match alternatives (esp if you've tried them in electric)#2023-07-1317:19Dustin Getzwe haven't checked any of them#2023-07-1317:20s-olalso taking general suggestions then, not finding much tbh since core.match seems to be getting traction#2023-07-1317:22Vincentlearning how core.match works via errors is... interesting xD#2023-07-1317:24s-oltrying https://github.com/dcolthorp/matchure, will report back#2023-07-1317:25s-ol> 13 years ago
😬#2023-07-1317:26Dustin Getzhttps://github.com/HealthSamurai/matcho#2023-07-1317:26Dustin GetzBut uses spec - we haven't checked that either#2023-07-1317:26Dustin Getzactually maybe we have, I think we do have stuff using spec actually#2023-07-1317:34s-olthat one doesn't do destructuring, which is half the benefit of using pattern matching for routing#2023-07-1317:34s-olmatchure also seems to do wonky things#2023-07-1317:34s-olno big deal, I'll go back to hand-rolled logic here#2023-07-1317:35s-ol(cond and let)#2023-07-1317:35Dustin GetzDo you have a one-liner demonstrating the requirement wrt destructuring, do you just mean this : [:account type id]#2023-07-1317:36s-olthat#2023-07-1317:36s-oli guess i can do a match and then a let with destructuring#2023-07-1317:36s-olbut at that point i can do the condition with cond and be as brief#2023-07-1317:39Dustin Getz(e/defn Todo-list []
(e/client
(dom/h1 (dom/text "core.match test"))
(dom/pre
(dom/text
(pr-str
(e/server
(match [:b]
[:a] "A"
[x] x
:else "fail")))))))
this much is working for me#2023-07-1317:39s-olhuh, interesting#2023-07-1317:39Dustin Getzi can rep your error, which is also a type error right?#2023-07-1317:40s-olcan or cant?#2023-07-1317:41Dustin Getz(match 123 x x :else "fail") works#2023-07-1317:41s-olI'm not sure? it doesn't say type error anywhere but it complains about something being a class rather than a constant#2023-07-1317:41s-olsee stack trace above#2023-07-1317:41Dustin Getz(match 123 [x] x :else "fail") is a weird compiler error#2023-07-1317:41s-olokay yes, that's what I saw as well#2023-07-1317:42Dustin Getzclj꞉user꞉> (require '[clojure.core.match :refer [match]])
nil
clj꞉user꞉>
(match 123 x x :else "fail")
123
clj꞉user꞉>
(match 123 [x] x :else "fail")
"fail"
at JVM repl this works#2023-07-1317:43Dustin Getzso it's confirmed an electric interaction afaict#2023-07-1317:58Dustin Getzwell the good news is I tested every usage in the basic tutorial and only one failed, the vector match#2023-07-1317:58Dustin Getzhttps://github.com/clojure/core.match/wiki/Basic-usage#2023-07-1317:58Dustin GetzSo my concerns about exception control flow seems to be not an issue#2023-07-1317:59Dustin Getzaaaand it worked in clojurescript (e/client), it only fails in e/server#2023-07-1318:00Dustin GetzI'll ticket it at this point, it is likely a simple bug in electric#2023-07-1318:15Dustin Getzminimal repro is (match [[1]] [[1]] 'pass), it's the matching of the vector type. (match [1] [1] 'pass) passes#2023-07-1318:15Dustin GetzDo you really need to match an inner vector? If your paths have static width i dont think you will hit this#2023-07-1318:16Dustin GetzThe core.match wiki implies this is just an optimization anyway for random access matching#2023-07-1318:21Dustin GetzFor matching dynamic args the sequence syntax works (as a workaround)
(match [[:foo :bar 1 2 3]]
[([:foo x & args] :seq)] args) ; works
; compare to
(match [[:foo :bar 1 2 3]]
[[:foo x & args]] args) ; crash
cursed syntax though#2023-07-1318:39Dustin GetzDon't you want the route matching to happen on the client anyway?#2023-07-1318:41Dustin Getzworks
(e/defn Baz [a] (dom/text "Baz" (pr-str a)))
(e/defn Bar [a] (dom/text "Bar" (pr-str a)))
(e/defn Todo-list []
(e/client
(dom/h1 (dom/text "core.match test"))
(dom/pre
(let [page [:foo :bar 1]]
(match [page]
[[:foo :baz a]] (Baz. a)
[[:foo :bar a]] (Bar. a)
:else (dom/text "404"))))))#2023-07-1510:22s-olhad moved it to server as a convenience since the views are server-colored so were all wrapped in e/server but yeah I'll try client!#2023-07-1510:22s-olaway from computers this weekend but I don't think I had nested vectors? iirc I had this issue with top-level vector matches#2023-07-1511:09Dustin Getzyou mean fixed arity tuples? (the core.match syntax is cursed, one or both of us may be confused)#2023-07-1511:10Dustin Getzin the matches, [] just means “pattern row” according to a footnote at the bottom of one of the wiki pages#2023-07-1512:23Dustin Getz> moved [router] to server as a convenience since the views are server-colored so were all wrapped in e/server but yeah I'll try client!
note that the variable inputs to the router are a user interaction on the client, so your DAG looks something like this: user event -> router -> views. So it doesn't seem to matter whether the router is on the client or server
i.e., both of these factorings are the same:
[client: event] -> [server: router -> views]
[client: event -> router] -> [server -> views]#2023-07-1512:24Dustin GetzNote that a UI may have client-only pages (i.e. storing local state in URL), in which case the factorings are no longer the same and co-locating the router with the user interaction becomes preferred to avoid a superfluous round trip#2023-07-1611:47s-olyeah, it has little impact in my case. With "for convenience" I guess I really meant brevity; having one e/server around the whole router block vs one e/server per each View being invoked#2023-07-1611:48s-olmight be the case that some network round trips could be eliminated but I'm not attempting any optimization at this point#2023-07-1611:49s-ol> in the matches, [] just means “pattern row” according to a footnote at the bottom of one of the wiki pages
>
hm, indeed I am confused now. Will get back to you about this when I next give it a go#2023-07-1715:32s-olokay, so: e/client (match) is working great, thanks for reminding me to try it out again.
the pattern vs pattern row thing is a bit confusing. I'm currently using a single [] syntax to match against a flat vector-of-strings value:
(e/server
(match hash
[:tx :new]
(new transaction/View "new-tx" {"new-tx" {:tx/label "new transaction"}})
;:tx/value-time (new js/Date)}})
;[:tx id]
;(new transaction/View (js/parseInt id) nil)
nil nil
:else
(new NotFound))
(match path
[:link :tink & rest]
(new tink/LinkAccount rest query)
[:account type id]
(new account/View (e/client (keyword (name type) id)) hash)
:else
(new NotFound))))))))))
this works e/client and fails in e/server with the IPersistentVector error posted above#2023-07-1715:34s-olin the wiki section on https://github.com/clojure/core.match/wiki/Overview#vector-matching they use [[a b]] which suggests that the "outermmost [] is special", but they also have an example with "https://github.com/clojure/core.match/wiki/Overview#matching-single-variables" without any wrapping which is odd again. I've tried doubling-up to (match [hash] [[*:tx :new*]] (new ... but am getting the same error on e/servre#2023-07-1715:39Dustin Getzi agree that the "single variable" section seems exceptional#2023-07-1715:39Dustin GetzI am confident enough now to just state that core.match has cursed syntax and sucks actually#2023-07-1409:20franky-dlouhanHi everybody! Has anybody tried out to run Electric with Mongo DB? Is that possible?#2023-07-1409:27braai engineerYes, Electric is not specific to any database. But if you want Electric to react to changes in Mongo, you either need to centralize changes (i.e. increment a counter every time you transact and pass the number down to your Electric components to trigger a re-render, or produce a Missionary stream using db.collection.watch): https://www.mongodb.com/docs/manual/reference/method/db.collection.watch/#2023-07-1419:07franky-dlouhanThank you, I'll check this out!#2023-07-1409:25braai engineerWhy am I seeing, INFO : Unserializable reference transfer: datomic.query.EntityMap {:db/id 17592186045430} given that entity does not cross the server/client boundary? Only v does, which is a string.
(e/defn TextField [eid attr]
(e/server
(let [entity (d/entity db eid) ;; hoist plz?
v (get entity attr)]
(e/client
(dom/div
(TextInput. (str v) ;; hmm
(e/fn [new-value]
(e/server
(d/transact conn [[:db/add eid attr new-value]])
nil))))))))#2023-07-1410:36Dustin Getzeither v is not actually a string (ref attr?) or the log is coming from another place in your code#2023-07-1410:37Dustin Getzafaict#2023-07-1409:52braai engineerWhat is this ClassList error about?#2023-07-1620:13Dustin Getzanother regression in class handling, (dom/props {:class ""}) crashes sorry#2023-07-1409:52braai engineerWon't this Reagent Interop example add two mousemove listeners when one would suffice? https://github.com/hyperfiddle/electric/blob/8ed95e899cc0d9a51dad3bad0f24c5614b13c8e2/src-docs/user/demo_reagent_interop.cljc#L68-L69#2023-07-1411:35Dustin Getzthe global singleton will be shared, only one listener is created#2023-07-1411:36Dustin Getz(singletons are idiomatic in FRP, it takes some getting used to – because as you note, instantiation is generally an effect)#2023-07-1419:12braai engineerAh, didn’t realize it was a singleton. Still feels like destructuring a single value (which contains both x & y) would be cheaper.#2023-07-1410:42braai engineerAnything I can do to speed up Electric compilation until incremental builds land? Will breaking things up into smaller namespaces help?#2023-07-1411:32Dustin GetzThe only workaround is to make unreachable parts of the app you aren't working on (i.e. hardcode the route statically or comment things out)#2023-07-1411:33Dustin GetzBreaking it up will not help, e/defn just quotes the electric code and sticks it on the var for a later full-program compile#2023-07-1415:07Dustin Getzoh you can also try a cloud ide setup if your local cpu is old#2023-07-1417:54Vincentlike Dustin said, if you reduce total lines of code by moving unused components to temporary other text files, you can compile fast, until inc builds#2023-07-1419:13braai engineerIs there a way to start two electric compilers in the same project that compile two different namespaces? So I can have quick updates for my "components" that I need quick iteration for, and slow compile for my “fat” app?#2023-07-1419:13braai engineer(Where components are shared)#2023-07-1423:46Vincentnot sure, hacks probably not the point of interest, but i think 2 projects (one lite one not) might be good here?#2023-07-1423:47Vincentif you can make a "minimal setup" and duplicate it, it might be a way to tackle this cleanly. or just duplicate current setup and take a machete to it / go full eraserhead#2023-07-1423:48Vincentin the same setup multiple compilers seems like an easi incidental comploexity jungle#2023-07-1501:18Dustin Getzyou cannot link across compilation units, however the same namespace can be referenced from two projects with separate compilation units#2023-07-1512:07Dustin GetzE.g. a "devcards" workflow may solve your problem – electric components ideally can stand alone if their dependencies are in scope. Now you have two builds - a devcards build (one component at a time) and a fat app build#2023-07-1512:07Dustin Getzif you trigger compiles manually for the fat app build (i.e. use shadow.api/compile rather than watch) they can probably even share REPL#2023-07-1502:04Drew Verleeis it advisable to include hyperfiddle.rcf tests in production code? I recall that it gets compiled out, but i can't find that in the docs.#2023-07-1502:05Dustin Getzrcf tests are comments unless explicitly turned on#2023-07-1502:08Drew VerleeTy Dustin. Hope things are going well on your end :)#2023-07-1502:05Drew Verleeit says it right at the start of the usage section.#2023-07-1514:49euccastrois there any well-known situation where electric would raise a electric_client.cljs:67 WebSocket is already in CLOSING or CLOSED state. error when trying to return a list of results from the server?#2023-07-1515:33Dustin GetzI don't recall, it sounds like a server crash. I don't recall any known server crashes either, there is a catch-default-and-log at the userland entrypoint. A user had a server crash earlier this week which was the result of mismatched client/server assets (due to connecting to the prod websocket url from a live local shadow build - big no no)#2023-07-1517:23euccastroOK, I'll try and narrow it down to a small reproduction snippet#2023-07-1519:16Geoffrey GaillardGiven my understanding of the electric websocket impl, it could be that the client is trying to send a message to the server at the same time the server has closed the socket (for whatever reason).
Meaning the client tries to send a message after the socket close but before the websocket onclose callback is called.#2023-07-1519:42Dustin Getz@U65FN6WL9 also make sure you're watching server logs, there should be a stacktrace in the log#2023-07-1519:43Dustin GetzI added documentation about the logger config to the starter app readme last week - https://github.com/hyperfiddle/electric-starter-app#2023-07-1521:03Dustin GetzMaybe it can be a busted shadow build#2023-07-1521:03euccastroI have an app made from the electric-starter-app that is working fine in the dev environment but when I make an uberjar following the README instructions I get a clojure.lang.ExceptionInfo: Unable to resolve symbol: nsqc/select-users, where nsqc is a Clojure-only namespace that I added at src/ (i.e., a sibling of electric_server_java8_jetty9.clj , user.clj, etc.), and which I import in a #?(:clj (:require [nsqc] ,,,)) clause in the app .cljc. Should I do something differently so it will be packed/found correctly?#2023-07-1521:03Dustin Getzyou shouldn't have to do anything special, please post your namespace as a gist so i can take a look#2023-07-1521:04Dustin GetzThis is a build-time error?#2023-07-1521:05euccastrono, it happens at startup, after the :point_right: App server available at printout. I'll paste my gist#2023-07-1521:07euccastroIt requires other files that are all somewhere under /src#2023-07-1521:10Dustin Getzah sorry i meant the other namespace that depends on this one#2023-07-1521:12euccastroI'm doing things in an unorthodox way by trying to keep most state in the client. I tried to have things server side but then I stumbled into the error in the previous message, which as I said I'll try and get a small reproduction for#2023-07-1521:14Dustin Getzthis may be dumb but in clojure are you allowed to nest :requires like that ?#2023-07-1521:15euccastroit works in the dev environment, but I can see how that could confuse the build tools#2023-07-1521:16euccastroI'll try making this a cljc and requiring it unconditionally#2023-07-1521:16Dustin Getzi get this:#2023-07-1521:16Dustin GetzIf that error is somehow being swallowed in dev it would also explain your server crashes#2023-07-1521:18euccastrono, the :requires are not nested in the source; they're siblings. they look nested because of the extra indentation of the ?#(:clj form#2023-07-1521:18Dustin Getzoh sorry#2023-07-1521:18euccastrothat namespace is required successfully and used extensively in the dev environment#2023-07-1521:21Dustin Getzit's a single segment ns though, so (nsqc/foo) is fully qualified even if the :require is absent#2023-07-1521:22Dustin GetzI'm looking for reasons the uberjar build could fail, this area of exploration smells right#2023-07-1521:22Dustin Getztry this alternate syntax
(ns app.core
(:import [hyperfiddle.electric Pending])
(:require
[clojure.pprint :as pprint]
[clojure.set :as set]
[clojure.string :as string]
[clojure.walk :as walk]
[contrib.str :as stringc]
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?@(:clj [[ldap]
[nsqc]]
:cljs [[cognitect.transit :as transit]
[com.cognitect.transit.types :as transit.types]])))
#2023-07-1521:23Dustin Getzyou could be right that the build tooling is parsing it or something, idk#2023-07-1521:28euccastrosame error; I just noticed it only happens when I try and open the app in my browser, not at app start itself#2023-07-1521:32Dustin Getzcould the namespace be failing to load? At runtime, that error just means the namespace isn't loaded, the error came from Clojure, Electric did the right thing trying to resolve that symbol#2023-07-1521:33Dustin Getztry a (println "loaded nsqc") at the bottom#2023-07-1521:34euccastroI'll also try requiring it from user.clj without the #? machinery#2023-07-1521:34euccastrothat may nudge shadow to include it#2023-07-1521:35euccastroI see the "loaded nsqc" printout at build time#2023-07-1521:36euccastroI guess I shouldn't see it at runtime if this is AOT compiled#2023-07-1521:36Dustin Getzyeah, hmm#2023-07-1521:37euccastroyes, no printout and same exception; I'll try requiring it from user.clj#2023-07-1521:41Dustin Getzprod.clj#2023-07-1521:42euccastronow I get the printout twice at build time, once at runtime, and I get a new crash even before I try and connect a browser:
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/core/cache/CacheProtocol#2023-07-1521:42euccastroI'll try prod.clj#2023-07-1521:44euccastroOh, I was still requiring app.todo-list instead of my app.core namespace in prod.clj. That will mess with things I guess. I'll try and fix that first#2023-07-1521:44Dustin Getzah#2023-07-1521:44Dustin Getzi should have just asked you to add me to the repo, i know to check for that#2023-07-1521:47euccastroyes, that was it, sorry for the noise and thanks a lot for the help!#2023-07-1606:38VincentHi, I have a value on the server "customer count" that i want to live stream to the client, can i e/watch a value defined in the #?clj side?#2023-07-1610:46Dustin Getzyou can watch an atom on the server, what is streamed is electric scope#2023-07-1613:16braai engineer@U055PQH9R4M:
#?(:clj (defonce !customer-count (atom 0)))
(e/def customer-count (e/server (e/watch !customer-count)))
(e/defn App []
(e/client
(dom/text "Count: " customer-count)
(ui/button (e/fn [] (e/server (swap! !customer-count inc) nil))
(dom/text "increment customer"))#2023-07-1615:48Vincentthank you. I need to persist this value between server restarts and wonder what the simplest way is#2023-07-1615:55Dustin Getzsave to disk when it changes (you can use electric for that), read from disk on startup. World's dumbest database#2023-07-1615:55Dustin GetzBetter: https://github.com/jimpil/duratom (search around for "duable atom clojure")#2023-07-1615:56Dustin GetzI've not tried that, was the first google result#2023-07-1616:06Vincenti actually went to duratoms last night, i might try again this morning with a clearer head 😅#2023-07-1616:22VincentDuratom serializes via pr-str not sure if that is problematic#2023-07-1616:23Dustin Getzpr-str correctly turns values to edn for all "simple" datatypes that we commonly think of as "EDN"#2023-07-1618:53braai engineer@U055PQH9R4M Datalevin or Datomic.#2023-07-1621:08Vincentty both y'all.
am already using xtdb so we'll see if i can figure out how to watch a single integer 😂#2023-07-1608:14braai engineerAnyone have a Firestore integration lying around for Electric, please? So that e.g. components can conveniently subscribe to a Collection, Query or Document?#2023-07-1611:53Dustin GetzCollect the appropriate subscription APIs and we can probably whip up the missionary expression quickly#2023-07-1612:30braai engineerI’m sure this has been asked already, but how can I combine Electric with an existing Ring application? Ideally Electric could be a Ring handler that I drop into an existing app. In meantime will deploy on subdomain, but would like certain parts of app to be Electric, and other parts static or using other frameworks.#2023-07-1612:32braai engineerHmm, OK I see it’s mainly the /ws Websocket handler + the resources handler. I’m sure I could figure it out, but a working example would save time.#2023-07-1612:57Dustin GetzI think it is not quite that simple, the websocket handler is http server implementation dependent iiuc and changes whether you use httpkit, jetty9, jetty10 etc#2023-07-1612:59Geoffrey GaillardThe electric starter app has an https://github.com/hyperfiddle/electric-starter-app/blob/e38c61b66727f82d665fd2b1ff11b11afdce2264/src/app/electric_server_java11_jetty10.clj#L88 for jetty.#2023-07-1613:00Dustin GetzAlso here is a https://github.com/hyperfiddle/fulcro-electric-template/blob/47e9340f7b9f69759a5f58831eb915b06b71e54d/src/main/app/server_components/middleware.clj#L93, i linked to the line that is http implementation dependent#2023-07-1613:02Dustin Getzif you could specify exactly what it is that you want, in a way that is not under-specified (i.e. the requirements are complete and implementable), that would help us provide the right example. Specifying the requirements here is the hard part#2023-07-1613:46braai engineerI’ll use whatever version of Jetty Electric wants, or http-kit if that’s easier. I would like an /api endpoint that isn’t tied to Electric that handles some inbound JSON POST requests and transacts something to Datomic and responds with some JSON. Eventually I can move this out to a separately deployed project but right now I just want one deployable artifact and to work in one project folder without having to compile two things. The most important thing for my weekend projects is having super fast iteration and deployment.#2023-07-1613:51braai engineerCan I just add another route under wrap-demo-router next to /auth to deal with my API requests?#2023-07-1613:54Dustin Getzstarter app demonstrates unmanaged endpoint – "/auth"#2023-07-1614:31Dustin GetzYes, there is no need to separate into multiple projects (we would consider that an anti-pattern, we want the electric server to be able to reach as much of your classpath as possible)#2023-07-1715:38telekidI'm about to try to embed an https://github.com/asciinema/asciinema-player in an electric app and have a couple of questions.
1. The player has an imperative API (methods like play, pause, seek.) What is the idiomatic approach to interop here? Would it be to store player state in an m/watched atom (`{:status :paused}` ) and invoke the imperative APIs as that atom updates?
2. ~The player API's ~#2023-07-1716:01telekidUpdate: I figured out #2 after finding similar examples in slack history. Not sure if it's idiomatic or not though – will post my code here in a bit. The thing that was unintuitive is that you can run arbitrary code at the end of a (dom/div ... ) expression, and that code will be evaluated after the element has been appended to the dom.#2023-07-1717:16Dustin Getzyes the approach for #1 that you described is fine, to do better i would need to dig into low level detail of the imperative api. is it async, can it fail, are there callbacks, etc#2023-07-1717:18Dustin Getz(case status :paused (.pause ref)) is acceptable, it will run the effect on transition but never run the effect twice in a row#2023-07-1718:04HendrikWhile dealing with imperativ stateful APIs I had the following problem: The state changed silently and there was no way to register a callback to a state change. In case of asciinema-player this could probably happen if track finishes and the state changes to stopped (just guessing here, I do not know the API). Anyway, my solution was to use some polling triggered by a clock:
(if (= "visible" e/dom-visibility-state)
(new (m/sample #(pollstate player-ref) e/<clock))
(throw (Pending.)))#2023-07-1718:06Dustin Getzwhat do you mean “state changed silently”#2023-07-1718:07HendrikThe state of an obj is changed by some API internals.#2023-07-1718:09HendrikOne way: A method-call internally creates a timer and mutates itself in the callback.#2023-07-1718:09Dustin Getzah the foreign reference, got it#2023-07-1718:10Dustin Getzobject.observe might be better than polling #2023-07-1718:21Hendrikmaybe. it could be triggered often, if the obj does a lot of member variable setting internally. It depends on the use case and its semantics (streams vs signals 🙂 )#2023-07-1719:31Dustin Getzm/relieve will immediately relieve backpressure on the observe's callback if it fires rapidly (and then Electric will work skip from there same as with the clock), i just think it will be cheaper than the clock which is firing at browser animation rate, you can probably see the clock in your CPU usage (unless in your case you are doing 3d animations anyway so the cpu is constantly on anyway)#2023-07-1719:33Dustin GetzIm actually confused a bit now about the cost of the clock; in the two clocks demo it is probably the streaming of the clock over network (io/serialization/radio on mobile) that causes noticible cpu (like 5%)#2023-07-1719:35Dustin Getzas long as you're looking at the tab and interactive, I think our guidance is that it is free, the cpu is underutilized and should be used as long as it stops when the user looks away#2023-07-1719:35Dustin Getzso your approach is fine#2023-07-1720:05telekidhere's where I've landed:
(ns hypo.server.asciinema
(:require [hyperfiddle.electric :as e]
[missionary.core :as m]
[hyperfiddle.electric-dom2 :as dom]
#?(:cljs ["asciinema-player" :as ap])))
(defn -listener [player listener-name]
(fn [!]
(let [cb (fn [] (! (keyword listener-name)))]
(.addEventListener player listener-name cb)
#(.removeEventListener player listener-name cb))))
(defn create-player [dom-id cast-url]
(let [node (.getElementById js/document dom-id)
player-obj (ap/create cast-url node)
!player-obj (atom player-obj)
player-obj-f (m/watch !player-obj)
playing-f (m/observe (-listener player-obj "playing"))
pause-f (m/observe (-listener player-obj "pause"))
playback-f
(->> (m/ap (m/?> (m/amb= playing-f pause-f)))
(m/reductions {} :pause)
(m/relieve {}))]
(m/latest vector player-obj-f playback-f)))
(e/defn Player []
(let [dom-id (gensym)
cast-url ""]
(dom/div
(dom/props {:id dom-id})
(let [[player-obj-f playback-f]
(new (create-player dom-id cast-url))]
(e/on-unmount #(.dispose player-obj-f))
playback-f))))
Player returns a flow of playback events which can be observed by the caller. Not sure if that's idiomatic?
Next, I'll be adding an argument to Player that accepts a flow of events, which will be used for external control of the player.#2023-07-1720:17Dustin GetzYeah, returning the joined flow is idiomatic or at least we'd like it to be, please ask if you hit any gotchas#2023-07-1807:14henrikAFAIK, object.observe is deprecated and probably shouldn’t be relied upon. The suggested replacement is Proxy, which allows you to produce a new object where you’ve (tediously) redefined or wrapped some or all properties on the original object.
Aside from this and polling, I’m not aware of a better way to listen for changes on a plain JS object.#2023-07-1810:27Dustin Getzthanks i didn’t know that#2023-07-1721:48VincentLately I have been running into this xtdb issue more frequently,
I wonder if anyone else sees it ever. I hit it when developing locally and overwrite entries in xtdb or delete#2023-07-1721:54Dustin Getzplease move long pre to the thread and add as attachment - this will render it better on mobile#2023-07-1721:56Dustin Getzi will probably need to see the code can you add me to the repo#2023-07-1721:57Vincentroger that x2#2023-07-1721:57Vincent#error {
:cause nil
:via
[{:type java.lang.InterruptedException
:message nil
:at [xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]}]
:trace
[[xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]
[xtdb.index$layered_idx__GT_seq$step__11587 invoke index.clj -1]
[xtdb.index$layered_idx__GT_seq$step__11587$fn__11601 invoke index.clj 315]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 58]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$seq__5467 invoke core.clj 139]
[xtdb.query$compile_find$fn__14793$iter__14796__14800$fn__14801 invoke query.clj 1900]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$take$fn__5978 invoke core.clj 2901]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.Cons next Cons.java 39]
[clojure.lang.RT next RT.java 713]
[clojure.core$next__5451 invokeStatic core.clj 64]
[clojure.core$dorun invokeStatic core.clj 3143]
[clojure.core$doall invokeStatic core.clj 3149]
[clojure.core$partition_all$fn__8629 invoke core.clj 7338]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$map$fn__5935 invoke core.clj 2763]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$map$fn__5935 invoke core.clj 2763]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$apply invokeStatic core.clj 662]
[clojure.core$mapcat invokeStatic core.clj 2800]
[clojure.core$mapcat doInvoke core.clj 2800]
[clojure.lang.RestFn invoke RestFn.java 423]
[xtdb.pull$__GT_pull_result invokeStatic pull.clj 197]
[xtdb.pull$__GT_pull_result invoke pull.clj 188]
[xtdb.query$query$fn__14889 invoke query.clj 1980]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 58]
[clojure.lang.RT seq RT.java 535]
[clojure.lang.SeqIterator hasNext SeqIterator.java 38]
[xtdb.io.Cursor hasNext io.clj 282]
[clojure.lang.RT chunkIteratorSeq RT.java 517]
[clojure.core$iterator_seq invokeStatic core.clj 5754]
[clojure.core$iterator_seq invoke core.clj 5746]
[xtdb.query.QueryDatasource q_STAR_ query.clj 2050]
[xtdb.api$q invokeStatic api.clj 413]
[xtdb.api$q doInvoke api.clj 407]
[clojure.lang.RestFn invoke RestFn.java 442]
[app.todo_list$the_mine_user_record invokeStatic todo_list.cljc 1142]
[app.todo_list$the_mine_user_record invoke todo_list.cljc 1141]
[app.todo_list$eval55852$fn__55853$fn__55854$fn__55855 invoke NO_SOURCE_FILE 0]
[clojure.lang.AFn applyToHelper AFn.java 152]
[clojure.lang.AFn applyTo AFn.java 144]
[clojure.core$apply invokeStatic core.clj 667]
[clojure.core$apply invoke core.clj 662]
[app.todo_list$eval55852$fn__55853$fn__55854 doInvoke NO_SOURCE_FILE 0]
[clojure.lang.RestFn invoke RestFn.java 397]
[missionary.impl.Thunk run Thunk.java 58]
[java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1136]
[java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 635]
[java.lang.Thread run Thread.java 833]]}
#2023-07-1722:38Dustin Getzi think 1) we need to wrap your queries with e/offload #2023-07-1722:39Dustin Getzare they already wrapped in e/offload?#2023-07-1722:40Dustin Getz2) is this error just in the log or is the app crashing too or what? what happens if you simply silence the error with try/catch#2023-07-1722:42Dustin Getzif the reactive views update faster than the queries can keep up with, electric will interrupt the stale query by triggering this exception, so that the more recent query can start sooner#2023-07-1722:43Dustin Getzi’m surprised the exception is seen in userland though#2023-07-1723:45Vincentonce it happens, like once a value is changed, the [same] value being displayed goes blank (instead of being a number or string)
i have the queries wrapped in e/offload but nested in some e/client e/server blocks, i think that's ok?
i wonder if this idx__GT is from sort-by >
#error {
:cause nil
:via
[{:type java.lang.InterruptedException
:message nil
:at [xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]}]
:trace
[[xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]
[xtdb.index$layered_idx__GT_seq$step__11587 invoke index.clj -1]
[xtdb.index$layered_idx__GT_seq invokeStatic index.clj 325]
[xtdb.index$layered_idx__GT_seq invoke index.clj 302]
[xtdb.query$build_sub_query$fn__14491 invoke query.clj 1681]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]#2023-07-1723:48Vincentaktually i do not e/offload all of them#2023-07-1801:03Vincenttry/catch seemed ineffectual
rules around e/offload?#2023-07-1801:05Dustin Getzno special rules that i recall, i think i need to see your repo. make sure e/offload is added to the queries so i can grep for it#2023-07-1801:06Dustin Getzi should be able to manifest the issue in the xtdb starter app#2023-07-1801:07Dustin Getzdid you do anything in particular recently that made the issue worse, like for example persist an atom with a counter into the db?#2023-07-1801:30VincentFWIW i think it is just for local builds. my online build doesn't exhibit this behavior, presumably thanks to latency#2023-07-1801:31Vincenti recently gutted my 2500 line app to 1300 lines 😅
so it could have been many things laughcry#2023-07-1801:31Vincentbut again, thinking only offline dev mode is affected. will let you know if i see it online#2023-07-1801:32Dustin Getzok that supports my theory but still it is concerning#2023-07-2004:19Vincentseeing it sometimes on my production box, i think it's probably "issue between chair and keyboard" 😅
but not sure yet what is causing it
#error {
:cause nil
:via
[{:type java.lang.InterruptedException
:message nil
:at [xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]}]
:trace
[[xtdb.index$layered_idx__GT_seq$step__11587 invokePrim index.clj 306]
[xtdb.index$layered_idx__GT_seq$step__11587 invoke index.clj -1]
[xtdb.index$layered_idx__GT_seq invokeStatic index.clj 325]
[xtdb.index$layered_idx__GT_seq invoke index.clj 302]
[xtdb.query$build_sub_query$fn__14491 invoke query.clj 1681]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$seq__5467 invoke core.clj 139]
[xtdb.query$compile_find$fn__14793$iter__14796__14800$fn__14801 invoke query.clj 1900]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$partition_all$fn__8629 invoke core.clj 7337]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$map$fn__5935 invoke core.clj 2763]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$map$fn__5935 invoke core.clj 2763]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 51]
[clojure.lang.RT seq RT.java 535]
[clojure.core$seq__5467 invokeStatic core.clj 139]
[clojure.core$apply invokeStatic core.clj 662]
[clojure.core$mapcat invokeStatic core.clj 2800]
[clojure.core$mapcat doInvoke core.clj 2800]
[clojure.lang.RestFn invoke RestFn.java 423]
[xtdb.pull$__GT_pull_result invokeStatic pull.clj 197]
[xtdb.pull$__GT_pull_result invoke pull.clj 188]
[xtdb.query$query$fn__14889 invoke query.clj 1980]
[clojure.lang.LazySeq sval LazySeq.java 42]
[clojure.lang.LazySeq seq LazySeq.java 58]
[clojure.lang.RT seq RT.java 535]
[clojure.lang.SeqIterator hasNext SeqIterator.java 38]
[xtdb.io.Cursor hasNext io.clj 282]
[clojure.lang.RT chunkIteratorSeq RT.java 517]
[clojure.core$iterator_seq invokeStatic core.clj 5754]
[clojure.core$iterator_seq invoke core.clj 5746]
[xtdb.query.QueryDatasource q_STAR_ query.clj 2050]
[xtdb.api$q invokeStatic api.clj 413]
[xtdb.api$q doInvoke api.clj 407]
[clojure.lang.RestFn invoke RestFn.java 425]
[app.todo_list$latest_addition_track_records invokeStatic todo_list.cljc 1178]
[app.todo_list$latest_addition_track_records invoke todo_list.cljc 1177]
[app.todo_list$eval55433$fn__55434$fn__55435$fn__55436 invoke NO_SOURCE_FILE 0]
[clojure.lang.AFn applyToHelper AFn.java 152]
[clojure.lang.AFn applyTo AFn.java 144]
[clojure.core$apply invokeStatic core.clj 667]
[clojure.core$apply invoke core.clj 662]
[app.todo_list$eval55433$fn__55434$fn__55435 doInvoke NO_SOURCE_FILE 0]
[clojure.lang.RestFn invoke RestFn.java 397]
[missionary.impl.Thunk run Thunk.java 58]
[java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1144]
[java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 642]
[java.lang.Thread run Thread.java 1623]]}#2023-07-2506:06Vincent(try .. (catch InterruptedException e)) helped keep numbers displaying, i still get an exception but the program is not impeded at all so that's good. yeah maybe it's like a cascade of queries, maybe the way my view is drawn is making some redundant asks or somet#2023-07-2506:06Vincentty for your help#2023-09-2907:36wei+1 for this problem and solution. Just encountered an InterruptedException executing an xtdb query, and adding a try/catch seems to keep execution going without issues. However, I'd like to understand the root cause in case I'm papering over something important.#2023-09-2908:30xificurCif you can describe the case when it happens, provide the stacktrace etc. we can take a look. Maybe asking in #CG3AM2F7V will also yield better results#2023-09-2910:20Dustin Getzi suspect (as previously) that if you type fast and trigger query refreshes in response to typing, it’s possible to type faster than the query can finish, so Electric will (correctly) interrupt the no-longer-interesting query so that the more up to date query can run sooner#2023-09-2910:21Dustin GetzTry to repro in the starter app by putting a contrived slow query in it. Or “type fast” by using missionary to smash the query rapidly with m/sleep and/or m/seed#2023-09-2910:23Dustin Getzit’s missionary that is supervising the queries not electric fwiw, also i think m/via (inside e/offload) is pet of the repro #2023-09-2910:23Dustin Getzpart*#2023-09-3011:15weii tried pressing enter repeatedly (starting multiple queries) and couldn't trigger it. will report back if i find a repro#2023-09-3014:27Vincentit happens when i trigger xtdb/put by clicks#2023-09-3014:27Vincent(enter might actually be too slow to see the issue)#2023-09-2907:36wei+1 for this problem and solution. Just encountered an InterruptedException executing an xtdb query, and adding a try/catch seems to keep execution going without issues. However, I'd like to understand the root cause in case I'm papering over something important.#2023-07-1814:01Hendrik@dustingetz fyi I finally have a working Electric wrapper for three.js 🙂 Thank you so much for your help and advice !
https://github.com/HendrikLevering/electric_three_demo#2023-07-1814:01Dustin Getzill check it out, what are your thoughts?#2023-07-1814:11HendrikI think that Electric and three_js fit quite nicely. Three.js requires you to manually cleanup GPU resources. Electric’s RAII semantics are great to automatically cleanup resources. And I used three.js raycaster to get mouse events on scene graph objects which you subscribe too similiar to dom/on and dom/on! And a rerender only happens if the scene, camera or dom-size changed. Electrics caching and minimal recompute does a great job.#2023-07-1814:11Dustin Getzoh wow, that is unexpected#2023-07-1814:26HendrikSome things which are a little bit hacky at the moment:
orbit controls and camera. OrbitControls are a mutable JS blob which mutates the camera. This is not working completly declaritive, but I didn’t want to reimplement OrbitControls, which is 1000+ lines of mutable JS.
You have to require three.js in everyfile, where you use the wrapper. I think that this is solveable, but I surely reached my Macro Skills limit on this 😄
Three.js interfaces are inconsistent. Often you simply can set properties. However, there are some classes that require constructor args and sometimes there are setter functions. Especially the constructor args can be a little bit annoying. First of all, you cannot refer to this (like dom/props can) and secondly you have to account for this in the wrapper. I created the convention, that the first argument is a list of constructor args.#2023-07-1814:28HendrikIn summary I am quite satisfied with the result.
Now I can write code like this:#2023-07-1814:28Hendrik(e/client
(dom/div
(let [!state (atom 0)
state (e/watch !state)]
(ui/button (e/fn [] (swap! !state inc)) (dom/text "inc"))
(ui/button (e/fn [] (swap! !state dec)) (dom/text "dec"))
(dom/div
(dom/style {:background-color "blue"})
(dom/props {:class ["full-height"]})
(th2/canvas
(th2/WebGLRenderer [])
(let [camera (th2/PerspectiveCamera [75 th2/view-port-ratio 0.1 1000]
(th2/props {:position {:z 5}}))]
(th2/OrbitControls [camera dom/node]
(th2/control))
camera)
(th2/Scene []
(th2/AmbientLight [0xFFFFFF 0.2])
(th2/DirectionalLight [0xFFFFFF 0.3]
(th2/props {:position {:x 1 :y 1}}))
(when (> state -1)
(th2/Mesh [(th2/BoxGeometry [2 2 2])
(th2/MeshLambertMaterial []
(th2/props {:color {:r 0.0 :g 0.9}}))]
(th2/on! "click" #(println %))
(th2/props {:position {:z -2}})))
(th2/Group []
(th2/Mesh
[(th2/DodecahedronGeometry [1 0])
(let [h (th2/Hovered?.)]
(th2/MeshLambertMaterial []
(th2/props {:color {:r 0.5 :g (if h 0 1)}})))]))))))))#2023-07-1814:35Hendrik#2023-07-1915:21Dustin GetzI sent a PR to add your video and commentary to the readme, i'll put it on reddit after you merge to get some more feedback#2023-07-1915:21Dustin GetzAlso I'm going to tweet out the video, do you have a twitter for me to tag you?#2023-07-1915:21Dustin GetzGreat work#2023-07-1915:23Dustin GetzOne q i have is where is the boundary between the reactive clojure data and the underlying efficient vector implementation, is this just not an issue given we're maintaining a mutable scene graph?#2023-07-1915:24Dustin GetzLike if you want to animate the scene and the electric mouse signal is an input to it, how does the mouse signal end up in a matrix transform and whose job is it to coordinate that#2023-07-1915:26Dustin GetzAlso i recommend tonsky style indentation with electric (or at least it is the easiest way to get the parent-child maintenance macros to indent cleanly) - https://github.com/hyperfiddle/electric-starter-app/blob/main/.cljfmt.edn#2023-07-1915:35Dustin GetzHow many hours did it take you to do this?#2023-07-1916:50HendrikCool feel free to share. I haven’t had a twitter account, yet. So I created one @Leveringit
I did not count the hours. I would guess approx. 30 - 40.
I’ll check out the style indentation. Until now I used Calva’s default.
One q i have is where is the boundary between the reactive clojure data and the underlying efficient vector implementation, is this just not an issue given we're maintaining a mutable scene graph?
I’m not sure, if I understood this correctly. You are talking about the camera controls?
The problem with three.js camera controls is, that it mutates the camera directly. Moreover, I realized that camera matrix (which is internal derived camera state) should be updated immediatly after props change. It was done before in the render method, but raycasting depends on a correct camera, too. I wrote a little helper to do so.
Today I implemented my own version of OrbitControls, which was super easy with electric /missionary btw.
With this the flow becomes:
(let [!pos (atom {:x 0 :y 0 :z 5})
pos (e/watch !pos)
look_at {:x 0 :y 0 :z 0}]
...
(let [camera (th/PerspectiveCamera [75 th/view-port-ratio 0.1 1000]
(th/setter th/reset_camera th/set_camera pos look_at))
new-pos (ctrl/orbit-controls. camera {:rotateSpeed 1})]
(reset! !pos new-pos)
camera)
....
Now the boundary is on the constructor macro and the props settings (which marks a render flag internally)#2023-07-1916:58HendrikI am exited about how simple the orbit control implementation with electric and missionary is. Especially if you compare it against the original javascript implementation. And giving the controls some moment of inertia should be simple. Roughly: Generate a finite flow of declining values on mouse movement and make it debounced 🙂#2023-07-1815:29braai engineerPossibility of pre-rendering initial load on server then hydrating client for SEO & "cold start" performance? I know Electric aimed at rich enterprise apps, but we go to a lot of trouble to deploy Vite React app w/SSR to AWS Fargate just for SEO & performance, so if Electric was competitive with that...it would be an easier sell to the boss man.#2023-07-1815:30Dustin Getzplease use react for that#2023-07-1816:44braai engineerIs there a way to get a once-off HTML string render based on the current client/server DOM state? I.e. (e/render-html-string (ElectricDOMComponent.))?#2023-07-1817:35Dustin Getzno because electric-dom assumes e/client#2023-07-1817:36braai engineerso you'd have to do it on client with help of server?#2023-07-1817:48Dustin Getzelectric-dom is 300 loc (half of that is syntax macros); you will likely need to rewrite the 150 lines with the server rendering use case in mind#2023-07-1817:49Dustin Getzthis is very much not a priority for us#2023-07-1817:21telekidWhy does "V" print once but "R" prints once per tick?
(let [v (new (m/ap e/system-time-ms))]
(println "V" v))
(println "R" e/system-time-ms)#2023-07-1817:37Dustin Getze/system-time-ms is an electric value, missionary doesn't understand it and has resolved the underlying clojure var which is constant#2023-07-1817:38telekidElectric val i.e. e/def, right? Is it okay to create electric vals dynamically, e.g. (let [v (e/def ...]...) ?#2023-07-1817:39telekider, that probably doesn't make sense (e/def doesn't return anything.) Is there a different way to create electric vals dynamically?#2023-07-1817:50Dustin Getzlet and e/def, basically the same as clojure#2023-07-1817:50Dustin Getze/def is dynamic by default and works with binding#2023-07-1818:12telekidis it possible to convert an electric value to a missionary flow?#2023-07-1818:13telekidthanks for all of your help so far btw, Dustin. I've been waiting to play with HF since like 2018, it's really exciting to finally have something to experiment with#2023-07-1818:13Dustin Getzyes, wrap it in an electric thunk #2023-07-1818:14Dustin Getz(e/fn [] x) , surprisingly, typechecks as a missionary flow#2023-07-1818:14telekidah! okay cool#2023-07-1818:14telekidI would not have guessed that haha#2023-07-1818:14Dustin Getzthere are some examples in the main repo under src-docs in a subfolder somewhere#2023-07-1818:16Dustin Getznote- m/ap macro i think does not work inside electric code blocks due to a very low level issue (in Cloroutine which m/ap uses) that we haven’t fixed yet #2023-07-1817:46euccastroIt looks like you cannot use log functions from clojure.tools.logging directly in e/server blocks (but it is OK to log in functions called from them, even in the same namespace). Repro: add #?(:clj [clojure.tools.logging :as log]) to the requires of the app.todo-list namespace in the starter app, then insert a (log/info "hello!") in the e/server block in TodoItem . The app will fail to compile with
[:dev] Build failure:
------ ERROR -------------------------------------------------------------------
File: /home/es/git/electric-starter-app/src/user.cljs:7:1
--------------------------------------------------------------------------------
4 | hyperfiddle.electric
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
-------^------------------------------------------------------------------------
An error occurred while generating code for the form.
ExceptionInfo: failed compiling constant: app.todo-list; clojure.lang.Namespace is not a valid ClojureScript constant.
#2023-07-1818:57Dustin GetzYeah, sorry, it's a known issue and will be fixed#2023-07-1817:47euccastrobut if you make a helper like
(defn blah [x]
#?(:clj
(log/info x)))
and you call that from the e/server block, then everything works fine.#2023-07-1900:48VincentHi Can one put (e/server ..) blocks inside (e/client (.then promise ...) blockS?
like is nesting e/server into the promise's .Then ok?#2023-07-1900:49Dustin Getzno because .then takes a clojure fn not an electric fn#2023-07-1900:49Vincentok cool, i have pinponted my problem.#2023-07-1900:49VincentxD#2023-07-1900:49Dustin Getzyou can hack it with an atom, reset inside the callback. or you can use missionary#2023-07-1900:50Vincenti think atom is what i was reaching for intuitively. not sure how to be effective w/ missionary yet.#2023-07-1900:50Dustin Getzin #CL85MBPEF there is an await-promise operator iirc#2023-07-1900:51Dustin Getz)i mean in the channel)#2023-07-1902:46Vincentreset the atom inside a callback and then ... check the atom somehow for when it changes to do my (e/server side-effect? hmm. thinking#2023-07-1907:26Reut SharabaniI would like an example for it as well if possible. A lot of javascript libraries adopt the async/await way (and actually get rid of the sync operations) so I feel it would be valuable to document.#2023-07-1910:15Dustin Getzthere are several discussions , search m/observe#2023-07-1918:09VincentFigured it out, had to use a very simplified example, but basically i have a few atoms in a chain reaction#2023-07-1918:44Dustin Getzjust a heads up that chain reactions are an anti pattern, it will bite you eventually, try to think about ways to avoid that. centralizing your state into fewer atoms is one possible way, there are others#2023-07-2006:37tobiashere's the await promise gist:
https://gist.github.com/dustingetz/8823e47c13f780d18938363d1d641b5b
You can also turn it into a macro:
(ns app.await-promise
(:require [hyperfiddle.electric :as e]
[missionary.core :as m]))
(defmacro await-promise-macro [p]
`(new (e/task->cp (let [v# (m/dfv)]
(.then ~p
(fn [val#] (v# (fn [] val#)))
(fn [err#] (v# (fn [] (throw err#)))))
(m/absolve v#)))))
(e/defn Todo-list []
(dom/p (dom/text "Next bit of text is from a resolved macro: ")
(dom/text (await-promise-macro (.resolve js/Promise "hi")))))
Using it in an electric context you can now treat a promise as if it's just a value.#2023-07-1917:23telekidI've started a https://telekid.notion.site/Electric-Clojure-Missionary-2540f903263a4080b7a1bed3c7861023?pvs=4 documenting my learning experience. @dustingetz it contains a Feedback section where I'll try to note down anything that I've found confusing or unintuitive in case you're curious.#2023-07-1917:55Dustin Getzthank you this is valuable#2023-07-1922:42Vincentis there a way to avoid restarting the server clj -A:dev -X user/main after I add a new asset (e.g. jpeg) to resources/public ?#2023-07-1923:37Dustin Getzfirst of all you can .stop .start the server from a repl jacked into your editor, see user.clj#2023-07-1923:37Dustin Getzsecond, i’m shocked that you need a restart#2023-07-1923:37Dustin Getzit should just work#2023-07-1923:38Dustin Getzthe rebuilt javascript assets get picked up, for example, which is no different than a jpg#2023-07-2000:48Vincenthmmm ok#2023-07-2003:50Vincentmaybe just the nweb is strong with this one (moi) but i think i need to restart it for new .jpgs in /img thinking-face#2023-07-2003:56Vincentnevermind i think i just typed the wrong file name. i will check again later 😂#2023-07-2008:03HendrikI had some issues with advanced compilation. In #C6N245JGG theller noticed this shadow setting in the starter app:
:devtools {:watch-dir "resources/public" ; live reload CSS
:hud #{:errors :progress}
:ignore-warnings true} ; warnings don't prevent hot-reload
Is there a technical reason to ignore warnings or is it just user convenience?#2023-07-2008:49henrikI didn’t start from the starter app, but strapped Electric to an existing system. I didn’t set that one, and advanced compilation works fine for me.#2023-07-2008:56HendrikAdvanced compilation worked for me, too, after I had fixed the warnings. I was just wondering why we have this setting in the starter app.#2023-07-2008:57henrikAh, gotcha, I misread what you wrote.#2023-07-2010:08Geoffrey GaillardThis is historical. Electric used to produce a lot a false positive warnings in its early days. It might be time to revisit.
Note that shadow will prevent hot code reloading in presence of warnings by default. :ignore-warnings just allow hot code reload even if there are warnings. Warnings are still reported either way.#2023-07-2106:53HendrikI try to understand how the server side code is generated in prod setup:
Let’s look at the todo starter app:
(ns com.aeditto.electric.prod
(:gen-class)
(:require com.aeditto.electric.app.todo-list ; in prod, load app into server so it can accept clients
clojure.string
com.aeditto.electric.electric-server-java8-jetty9))
(def electric-server-config
{:host "0.0.0.0", :port 8080, :resources-path "electric/public"})
(defn -main [& args] ; run with `clj -M -m prod`
(when (clojure.string/blank? (System/getProperty "HYPERFIDDLE_ELECTRIC_SERVER_VERSION"))
(throw (ex-info "HYPERFIDDLE_ELECTRIC_SERVER_VERSION jvm property must be set in prod" {})))
(com.aeditto.electric.electric-server-java8-jetty9/start-server! electric-server-config))
I partially understand that the client is sending the program it wants to run.
However, where is all the server side code generated and stored (e/watch !db) and others?
For exampel I first eval a (e/defn SomeFn [] …) in a clj repl and then eval SomeFn var (gives unbound) and (meta SomeFn) which returns nil. Where is the server side code? It can’t be send by the client or is it?#2023-07-2108:54braai engineerDoes the client send the server the program it wants to run? @U09K620SG
If yes, how does security work?#2023-07-2111:49Dustin GetzYes, today the client uploads the Electric DAG. Compilation of the entire DAG (both client and server) happens in user.cljs. Yes, that means a malicious attacker could theoretically tamper with the DAG. This is already fixed in a feature branch by baking the DAG into the server at compile time. We haven't landed the fix on master yet because we wanted the hot code reloading issues to stabilize first. Also incremental compilation is about to change everything. In the meantime, if you are concerned you can assert the hash.#2023-07-2112:01henrik@U09K620SG Does “change everything” include significant API changes?#2023-07-2112:03HendrikHow would I get the hash and where would I check it? I assume the latter would be done in hyperfiddle.electric-jetty-adapter.electric-ws-message-handler , right?#2023-07-2112:10Dustin Getz@U06B8J0AJ no, just compiler internals and entrypoint internals#2023-07-2112:14Dustin Getz@U023LKF2PQV start here https://github.com/hyperfiddle/electric/blob/fe402e4c8383632240f3f20fa5802c83f51885c2/src/hyperfiddle/electric_jetty_adapter.clj#L88#2023-07-2112:15Dustin GetzI think the annoying bit is plumbing to inject compile time information to here without screwing up your hot code reloading#2023-07-2112:52HendrikThanks. That is the function I was referring to. You would check the hash before calling e/eval right? Now the question is: How do I get the expected hash which I can compare to the hash of the message?
Regarding hot code reload:
One solution could be:
provide the hash via an env var EXPECTED_HASH. Only make the hash comparison, if this env var is provided.
So we could get hot code reload for dev and security for prod.#2023-07-2113:03Dustin GetzYeah sth like that – it may not be worth it, as i said incremental compilation is coming and we'll likely address this at that time#2023-07-2117:00Dustin GetzNew essay, feedback please
https://dustingetz.electricfiddle.net/electric-fiddle.essay!Essay/electric-y-combinator (fixed)#2023-07-2117:00Dustin GetzPlease be pedantic, help make it bulletproof#2023-07-2117:01Dustin Getzmobile styles should work, pls report style issues#2023-07-2117:03refset> are ~is~ the exact primitive*s*#2023-07-2117:04refset> Using Y for recursion in Electric is very very bad
...because it's horribly inefficient, I'm assuming? Might be worth spelling it out#2023-07-2117:05Dustin Getzyeah, working on a clear answer to that thx#2023-07-2117:06refsetlove the ending 🙂#2023-07-2117:51telekidL7-9 using F as both the e/fn name and argument [F] makes it hard to understand what F is on L9 – you have to know enough clojure to realize that the argument binding will take precedence over the lambda's name#2023-07-2117:51Dustin Getzinteresting, i thought it was easier that way#2023-07-2117:52telekidI see the counterargument#2023-07-2117:52telekiddunno, might want another opinion#2023-07-2200:12tobiasNice! Maybe in the intro explain that the Y combinator isn't how you'd walk the file structure in practice, you're just doing it this way to demonstrate strong composition? I know you say that later in the essay but my first impression was that walking the file structure is harder than I thought it would be if it requires me to write a Y combinator.#2023-07-2320:00s-ollink is down, is that intentional?#2023-07-2320:00Dustin GetzSorry, it changed#2023-07-2320:00Dustin Getzhttps://dustingetz.electricfiddle.net/electric-fiddle.essay!Essay/electric-y-combinator#2023-07-2413:54Daniel JompheVery impressive and convincing for people already familiar with lambda fundamentals!
OTOH I realize that each one of those tutorial pages or essays should probably contain very visible links to the other ones, so that one might realize easily that (1) they’re deep, and (2) they should also check the basics.#2023-07-2308:41Lidor CohenIs there a recommended way (script?) of executing all my rcf tests? they're starting to pile up 🙂
P.S
I'm on cljs#2023-07-2309:44Geoffrey GaillardYes. You can build your cljs for the Karma test runner.
See https://shadow-cljs.github.io/docs/UsersGuide.html#target-browser-test.
In RCF repository, you’ll find examples of:
• a https://github.com/hyperfiddle/rcf/blob/7105b43231140de6f2f39ce38611c9f6c9dfc976/shadow-cljs.edn#L10
• https://github.com/hyperfiddle/rcf/blob/7105b43231140de6f2f39ce38611c9f6c9dfc976/karma.conf.js
• required https://github.com/hyperfiddle/rcf/blob/7105b43231140de6f2f39ce38611c9f6c9dfc976/package.json#L5-L8
• a https://github.com/hyperfiddle/rcf/blob/7105b43231140de6f2f39ce38611c9f6c9dfc976/ci/run_tests_browser.sh
◦ note the usage of the :test deps alias
◦ note :test alias has "-Dhyperfiddle.rcf.generate-tests=true"
#2023-07-2309:48Lidor Cohen🙏#2023-07-2319:57Jhttps://twitter.com/bholmesdev/status/1683178861150851072?s=46&t=FLgxpB8u-zgNc4wlhiD7Gw#2023-07-2419:23Dustin GetzI'm presenting Electric at London Clojurians tomorrow - https://www.meetup.com/London-Clojurians/events/294436947/#2023-07-2517:22braai engineerPlease record!#2023-07-2505:47Vincentwrapping some of my code in (try ... (catch InterruptedException e)) did help. :woman-shrugging:#2023-07-2511:55henrikHere’s a Thing:
(defn breakpoint
"Returns a flow of `k` when `selector` of `resolution` `unit` is matched,
or `nil` if it's not matched.
`unit` defaults to \"px\""
([selector k resolution]
(breakpoint selector k resolution "px"))
([selector k resolution unit]
(let [mm (.matchMedia js/window (str "(" selector ": " resolution unit ")"))
match (fn [^js obj]
(when (.-matches obj) k))]
(m/reductions x/discard (match mm)
(m/observe (fn [!]
(let [f #(! (match %))]
(.addEventListener mm "change" f)
#(.removeEventListener mm "change" f))))))))
(defn min-width
[k resolution-px]
(breakpoint "min-width" k resolution-px))
(defn media-queries
[& queries]
(m/relieve x/discard
(apply m/latest (fn [& values]
(into #{} (filter some?) values))
queries)))
So for example:
(e/def tailwind
"A flow of sets containing the currently activated Tailwind breakpoints."
(new (media-queries
(min-width :sm 640)
(min-width :md 768)
(min-width :lg 1024)
(min-width :xl 1280)
(min-width :2xl 1536))))
This is handy if you want to not only hide a subgraph when the window size shrinks, but also unmount it. Why calculate it at all if it’s going to be hidden?
matchMedia is preferred over observing the window width, since it sort of jacks into the CSS engine of the browser.
x/discard is equivalent to {}.#2023-07-2511:55Dustin Getzexcellent#2023-07-2516:31Dustin GetzZoom meetup in one hour! https://www.meetup.com/London-Clojurians/events/294436947/#2023-07-2519:30Dustin GetzIf you attended the talk please let me know how I did, here or by DM. Specifically interested in constructive criticism, we very much want to make this as good as we can#2023-07-2520:52jamesGreat talk, I learned a lot - you showed the depth of what you have doing and are going to do so I want to dig deeper. I'm not sure I would change anything for a Clojure-friendly audience. The work the team has done is really impressive.#2023-07-2523:27Danny AlmeidaAny chance this video is this talk will be made available as my timezone made it impossible for me to watch#2023-07-2523:31Dustin Getzvideo should be out shortly maybe tomorrrow#2023-07-2523:33Danny AlmeidaThat's awesome. I'm really interested in exploring electric after my failed attempts at learning Fulcro#2023-07-2523:33Danny AlmeidaHopefully, I'm not too dumb to pick it up 😀#2023-07-2523:37Dustin Getzi’m eager to learn how to make electric more accessible so please leave feedback, maybe i do another session#2023-07-2604:50Vincent@ULL659VU6 Electric is not difficult! Just a handful of "gotchas" and a bit of new pattern learning. I still do things in a very long, circuitous way and find a "short circuit" solution in Electric frequently, even after using it consistently ~3 months#2023-07-2604:52Danny Almeidathanks @U055PQH9R4M for sharing your experience. What would be the fastest way to get started ? I've been meaning to take it for a spin to see how the development experience is compared to say - Fulcro or Biff framework.#2023-07-2604:53Danny AlmeidaI love the idea of writing both server and client logic in one place - much easier on my brain to comprehend#2023-07-2604:55VincentTo be effective you do not need to know too much, other than perhaps the quirks around (dom/div (dom/props {:class "classname"}) (dom/text "text here"))
I think the best way to be effective fast is to try it yourself, use one of the example applications in the demo apps as a starting point and branch off it.
Taking a good look at the demo apps can get your creative mind going, and then if you see something close to a good starting point, or something you can expand from, go for it.#2023-07-2605:05VincentI have been thinking about doing a little youtube series on my dev setup. I'm still a newb so it would invite a lot of criticism 😅 but perhaps be helpful to others#2023-07-2605:14Danny Almeida@U055PQH9R4M that would be nice.. anything that helps someone learning it would be a positive thing in my humble opinion. I have learn't to ignore criticism and consider myself a noob when i'm learning anything new even if I have some previous experience in a related subject#2023-07-2607:59braai engineerMake all the text bigger, plz. I was watching on mobile most of the time and struggled to read any of the text.
'c' vs 's' on clock example, consider being more verbose with "server time" and "client time" or "server clock" and "client clock". Even though I know the example, I found myself having to resolve the "ref" of "c" to "client time" while listening.#2023-07-2610:18Dustin Getzthanks, didn’t realize ppl would view this on mobile i will adjust for that#2023-07-2610:34euccastroI only caught the latter part of the previous Scicloj meetup about Electric, but what I saw there struck me as more useful in order to get a mental model on how to use it. This talk had part of that as well, but my overall impression is that it was more about motivating the tech than helping convey the few major concepts and gotchas that would let you try it with success. Motivating the tech is a good thing to do as well, but I think it is easy to get excited about Electric and the major hurdle to adoption will be the worry that even if Electric had no bugs, when something doesn't work in your Electric program it will be much harder to debug because of the added layers of syntactic transformation and the async stuff going on behind the scenes.
So I think this is great for motivation, but a follow-up talk on the concepts and practicalities of how to develop Electric apps would be more impactful going forward. It could assume that one has followed the (excellent!) tutorials. It could present wrong or naive versions of the examples in the demo and explain how they are wrong or suboptimal, thus motivating the style in the tutorials.
For example, it could involve some of the common pitfalls (accidentally using a binding in the wrong context or otherwise accidentally triggering the attempt to send something unserializable through the wire -- IIRC some tutorial does this already?).
For a more subtle example, people used to existing CLJS SPA frameworks will want to keep as much state as possible in the client side and resent the components in hyperfiddle.electric-ui4 for forcing them to keep it server side instead. After having fought Electric to get it that way, with some success, I gained an appreciation for why server side state should be preferred.
When there is more than one way to do something, explain some of the reasons why you would prefer one vs the other in certain circumstances. E.g., what is the difference between (e/for ,,,) and (e/for-by identity ,,,)?
Also small examples on how to drop down to missionary and why you would want to do that (throttling, debouncing). What behavior to expect by default if you wrap dom inputs, and how do controls in ui4 currently override that (this may not be very worth getting deep into if you are working on ui5, but at least one example would help). Not going too deep perhaps, just introducing the topic and pointing to missionary tutorials for those interested.#2023-07-2610:37Dustin Getzgreat feedback#2023-07-2610:39euccastroTo motivate the last paragraph above: while the demos rightfully rely on fast client<->server turnaround, backoffice applications sometimes need to talk to DBs for which they are not the primary target, and thus run queries for which the DB schema is not optimized and not really worth optimizing, so some hints on using (or avoiding!) incremental search with slow backends (until ui5 solves that, I guess) would help.#2023-07-2612:57Dustin GetzPerhaps we should do a zoom Q&A for advanced users which I can then polish into a blog post#2023-07-2619:43Dustin GetzVideo https://www.youtube.com/watch?v=v-GE_P1JSOQ#2023-07-2715:16marcofisetJust watched the recording.
This is the most impressive piece of tech that I've seen in recent years, at least with regards to web development. Very excited to start playing with it.
It's totally beyond me how something like this is possible. I guess I got some reading to do 🙂
Amazing work Dustin 👏#2023-07-2715:51Dustin Getzit is a team effort!#2023-07-2717:46marcofisetYes of course! Congrats to the team too!#2023-07-2717:48marcofisetWhat are some resources that I should read in order to dive into the Electric codebase and get a sense of what is going on?
You've mentioned Missionary (?) as a library that Electric uses, also I guess I should read about DAGs too.#2023-07-2719:06Dustin GetzI recommend learning by doing
clone the starter app : https://github.com/hyperfiddle/electric-starter-app
go through the tutorial as needed : https://electric.hyperfiddle.net/#2023-07-2719:23VincentI have been using Electric to build an app for a few months I have no idea how missionary works 😅
Would recommend to focus on what you "need to know" for being productive#2023-07-2719:29marcofisetsure, I will definitely play with Electric first. I was just curious about what material I should be familiar with to be able to understand what's going on under the hood.#2023-07-2719:44VincentGreat question @U07SWUQTH
I think what helped me initially was knowing there's a "pending" state for values that are not-yet-determined / refreshed
and that we nest e/server and e/client blocks or e/client and e/server blocks (nesting can start with either is what i mean)
under the hood, i suppose it helps to know that the CLJS code will be rebuilt, but if you have an e/server or #:(clj ...)` code, you'll need to rebuild the server (restart)#2023-07-2719:45Vincenti'm still pretty new but i have made a good deal of progress in spite of my newbiness 😅#2023-08-0606:36siddharth yadavWatched the video today, excellent work by the team.
You mentioned how it is possible to push out the frontier of what is possible (PKM, fluent tools). I think having an example of that would be great, I think you don't have to write the PKM tool yourself there is already a codemirror example. Explaining that, would be a good tutorial or blog post because it seems to be at the right level of complexity in terms of breadth and depth needed to integrate such editors which is an essential part of PKM tools.#2023-07-2610:42braai engineerWould it be technically possible to pass binding vars as signal fn arguments instead of using dynamic bindings? I understand dynamic bindings are considered idiomatic in Electric, but this means there are two ways to pass references to "components" - in practice it's always conn or db for me, and I think it would be more ergonomic if that shared state was explicit.#2023-07-2610:45Dustin Getzcan you give an example#2023-07-2611:20s-ol@U051SPP9Z do you mean for the entrypoint? Everywhere else you can 'just do that' already. I'm not sure if there is a problem with scoping when you give the argument the same name as the binding variable defined at the namespace scope, but if so you can solve it by defining your db and conn in a different namespace and refering to them using that namespace only once in the entrypoint.#2023-07-2721:56Garrett HopperIs there a way to get clj-kondo to treat e/server and e/client as #?(:clj) and #?(:cljs) reader conditionals to avoid unresolved namespace warnings?
And potentially a way to get Clojure-LSP to behave similarly to get JS/JVM specific completions in context? (Cider is asking me "Language context at point" which is interesting)
(Just getting started, so forgive me if this is documented somewhere I haven't seen yet.)#2023-07-2722:03Dustin Getzi haven’t figured clj kondo out yet, i turned it off#2023-07-2722:03Dustin Getzi am feeling more fluent now in vscode though (i switched) so can possibly start figuring it out this week#2023-07-2722:04Dustin Getzi’m assuming it would be the same as emacs #2023-07-2722:07Garrett HopperI figure I probably could just add a reader conditional around every context switch, though that feels a bit verbose
Curious if the two could be done together somehow :thinking_face:
Thanks anyways; things are looking very exciting 😁#2023-07-2722:09lilactowna good question for #CHY97NXE2 - my guess is you'll need to write a custom hook. I'm not sure if custom hooks can control that context, though#2023-07-2722:23Dustin Getzthere are some old threads here as well with borkdude comments#2023-07-2800:26sergey.shvetsYou can completely ignore specific forms (and everything inside) like this:
$ clj-kondo --lint - --config '{:config-in-call {shadow.cljs.modern/defclass {:ignore true}}}' <<< "(require '[shadow.cljs.modern :as m]) (m/defclass 1 2 x y z)"
linting took 39ms, errors: 0, warnings: 0
Or if the form replicates some existing clojure form exactly you can do like that:
{:lint-as {shadow.cljs.modern/defclass clj-kondo.lint-as/def-catch-all}}}
I'm yet to try electric so not sure what will work better, but I think one of the approaches will work.#2023-07-2807:10xificurCsome previous discussions
https://clojurians.slack.com/archives/C7Q9GSHFV/p1681657983597109
https://clojurians.slack.com/archives/C7Q9GSHFV/p1681657983597109#2023-07-2909:03borkdudeI'm willing to look into this a bit, I'm not yet sure how support for electric would work#2023-07-2912:16Dustin Getzi think a lot of the lint rules simply do not apply (unless the strategy is changed)#2023-07-2912:34borkdudeOk we can simply shut off the linter in certain electric macros #2023-07-2912:44Dustin Getzthat seems to be about the same as shutting it off entirely, because the macros nest#2023-07-2912:45Dustin Getz(I think it's fine to shut it off entirely in Electric regions for a while while we learn more from experience, it will at least make the errors go away)#2023-07-2915:18JI use {:ignore [:unresolved-namespace]}#2023-07-2904:20JAtkinsCool thing:#2023-07-2904:21JAtkinsElectric + electron apps are gonna be fire. I threw this together in ~30m. https://github.com/JJ-Atkinson/electric-starter-app/tree/54c6f63c44cfe214b20efc133e8996653c5803a0#2023-07-2911:23Dustin Getzyes!!!!!#2023-07-2911:48Dustin GetzI get Missing client program manifest any idea why?#2023-07-2911:49Dustin Getzoh, missing package.json#2023-07-2912:04Dustin Getzalso missing file-information.edn what is this?#2023-07-2914:42JAtkinsRun bin/kondo#2023-07-2914:42Dustin Getzbin/kondo: line 2: clj-kondo: command not found#2023-07-2914:45JAtkinsAh, so clj kondo the linter. Im Surprised you don’t have it. I can put that file here temporarily. I’ll be back with it in 10m#2023-07-2914:45Dustin Getzit's integrated with vscode#2023-07-2914:46Dustin GetzI jsut don't want to guess what your dev setup might be#2023-07-2914:47Dustin Getzi will run `
brew install borkdude/brew/clj-kondo
`#2023-07-2914:47JAtkinsAh, yeah IJ + nix + a local repo with a bunch of project specific tools#2023-07-2914:47JAtkinsThat should do it#2023-07-2914:52Dustin Getzi have the app partially up now, there are compile time errors such as#2023-07-2914:53Dustin GetzCan you commit and push the version that ran?#2023-07-2914:54JAtkinsWeird it was running when I pushed. I’ll restart everything at try again#2023-07-2915:12JAtkinsOk give it a pull#2023-07-2915:12JAtkins5f58bc5dd1f6d45b16f8bfce732d295521ebef55#2023-07-2915:13JAtkinsI must've renamed the var just before pushing and not got an error, or I didn't see it.#2023-07-2915:41Dustin Getzit builds now but with javascript errors in console, do you have them also?#2023-07-2915:41Dustin Getzmain.js:1426 TypeError: add[0].set is not a function#2023-07-2915:43Dustin Getzi also had to add package.json from the ~starter~ examples app#2023-07-2915:44JAtkinsWeird. I don’t know what that js error is - can’t reproduce. #2023-07-2915:44Dustin Getzalso i get
core.cljs:3953 Cannot read properties of undefined (reading 'cljs$lang$applyTo')
in ( contrib.electric-codemirror/codemirror {:parent #object[HTMLDivElement [object HTMLDivElement]]} ) in contrib/electric_codemirror.cljc#2023-07-2915:45Dustin Getzwhen i click on a var to try to trigger "open files"#2023-07-2915:47JAtkinsre. package.json yeah that's my fault. I pulled a stupid and didn't add it to git. I'm clearing my node-modules now and restarting#2023-07-2915:47Dustin Getzwe might have mismatched package.json#2023-07-2915:47Dustin Getzi dont understand the cm error#2023-07-2915:47Dustin Getzits probably caused by the js error prior, which is in codemirror stack frame#2023-07-2915:48JAtkinsok package.json pushed#2023-07-2915:49JAtkinsif that doesn't resolve things I'm not really sure what will#2023-07-2915:52Dustin Getzis nix managing package.json?#2023-07-2915:52Dustin Getzor npm or yarn ?#2023-07-2915:52JAtkinsno#2023-07-2915:52JAtkinsjust making sure npm is on the path#2023-07-2915:53Dustin Getzworks now !#2023-07-2915:54JAtkinssounds like it was likely related to the package.json mismatch?#2023-07-2915:54Dustin Getzyeah#2023-07-2915:54JAtkinsyeah, my apologies. not sure why it wasn't already added to git but 🤷#2023-07-2915:57Dustin Getzno worries#2023-07-2915:57Dustin Getzdo you know why Kondo doesn't see the vars in the todo-app ns?#2023-07-2915:58JAtkinswhich ns? that file doesn't exist afaict#2023-07-2915:58Dustin Getzapp.todo-list#2023-07-2915:58Dustin Getz#2023-07-2915:58JAtkins#2023-07-2915:58Dustin Getzohhh#2023-07-2915:58Dustin Getztake 5#2023-07-2915:58JAtkinsit's a take 5 so the screen isn't just filled with vars#2023-07-2916:00JAtkinsIt's been a wacky couple months, but I think I've got time again to try out the pathom + electric stuff. will report back soon I hope with something interesting on that front#2023-07-2916:00Dustin Getzlooking forward to that!#2023-07-2916:34JAtkinsLol you caught me scrolling Twitter. Btw the reason the defn line wasn’t visible in one of the boxes is that you hadn’t rerun kondo and re loaded the slurped def#2023-07-2916:35Dustin Getzhuh#2023-07-2916:35JAtkinsKondo has line numbers but those change when you edit the file#2023-07-2916:35Dustin Getzoh did i f up the video#2023-07-2916:35JAtkinsNot really just mentioning it in case you didn’t see#2023-07-2916:35Dustin Getzi hadnn't, damn#2023-07-2916:36JAtkinsUp to you if ya wanna change it. I only noticed cuz I did the same thing a couple times last night. #2023-07-2916:37Dustin Getzwill you put this code in the public domain, i might run with it#2023-07-2916:39JAtkinsWhich code?#2023-07-2916:39Dustin Getzthe whole repo, i have to be careful with licenses#2023-07-2916:39Dustin Getzthe UI and source-parser#2023-07-2916:39Dustin Getzwe already have something similar I am just dotting is and crossing ts#2023-07-2916:39JAtkinsAh fair enough. Yes this repo is all oss#2023-07-2916:39JAtkinsI’ll be explicit in the gh fork#2023-07-2916:39Dustin Getzthanks#2023-07-2916:44JAtkinsok, added the unlicense#2023-07-2916:46JAtkinsIf it weren't such a dead end time sink, this would be pretty fun to turn into the lighttable code bubbles thing that never was implemented.#2023-07-2916:52Dustin Getzha yeah#2023-07-2914:33Joseph WinstonWarning -- New Electric User
I'm seeing errors like below. I thought that they were from me using try/catch but it is something different.
Ideas?
clj -A:dev -X user/main
Starting Electric compiler and server...
shadow-cljs - server version: 2.20.1 running at
shadow-cljs - nREPL server started on port 9001
[:dev] Configuring build.
[:dev] Compiling ...
[:dev] Build failure:
------ ERROR -------------------------------------------------------------------
File: /src/user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Encountered error when macroexpanding hyperfiddle.electric/boot.
Unable to resolve symbol: e/*http-request*
{:file "app/dashboard.cljc", :in [e/*http-request*]}
ExceptionInfo: Unable to resolve symbol: e/*http-request*
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:647)
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:632)
hyperfiddle.electric.impl.compiler/eval30807/analyze--30813/fn--30818/fn--30819 (compiler.clj:700)
hyperfiddle.electric.impl.compiler/eval30807/analyze--30813/fn--30818 (compiler.clj:699)
hyperfiddle.electric.impl.compiler/eval30807/analyze--30813 (compiler.clj:688)
hyperfiddle.electric/boot (electric.cljc:378)
hyperfiddle.electric/boot (electric.cljc:373)
...#2023-07-2914:35Dustin Getztry wrapping e/http-request in e/server#2023-07-2914:36Joseph WinstonCool -- Will do.#2023-07-2914:38Joseph WinstonSomething like this from the starter app? If that is correct, then II still have the problem
(let [session-id
(e/server (get-in e/*http-request* [:headers "sec-websocket-key"]))
api-key
(e/server (get-in e/*http-request* [:cookies "username" :value]))
]
...#2023-07-2914:39Joseph WinstonI think that I see the problem in my code. Let me try a few more things...#2023-07-2914:53Joseph WinstonStupid typo in a :require file (misnamed variable).#2023-07-2916:33Dustin GetzCleaned up CLJ-Kondo var explorer by @jatkin#2023-07-2921:15Reut SharabaniI want to supply a callback to cytoscape and I get an error when clicking (invoking e/fn callback):
n.call is not a function
My code:
(dom/div (dom/props {:id "cy"
:class "cy"})
(let [container (.getElementById js/document "cy")
opts (get-cy-opts nodes edges container)]
(.use cytoscape cy-elk)
(let [cy (cytoscape (clj->js opts))]
;; I guess error is in this callback
(.on cy "tap" "node" (e/fn [e]
(let [node-id (-> e .-target .id)]
(println "clicked" node-id)
(e/server (println "clicked" node-id)))))))
nil)
Is e/fn not interchangeable with functions (specifically for callbacks)?
How would I update the server that a selection was made? Specifically I want to update my db but I can't access it for updates (`!conn`) on the client afaict.
For now it just attempts to print what node was clicked on the client and switch to the server and print there as well.#2023-07-2921:28Dustin Getzyes e/fn and clojure.core/fn are different types#2023-07-2921:29Dustin Getzthe quick dirty way forward is to swap an atom in a clojure callback and then watch the atom from electric - this is the side effect way#2023-07-2921:30Dustin Getzthe better way is to adapt the event emitter into a missionary flow with m/observe and then use ‘new’ to await the missionary flow from electric - this is the functional way. search m/observe for several examples#2023-07-2921:31Reut SharabaniI thought about an atom side effect but was wondering if there is something simpler. I will search m/observe but I don't know anything about it.#2023-07-2921:32Dustin Getzatom way is 100% ok#2023-07-2921:46Reut Sharabaniactually I don't know how to do it. I know how to use an atom on the backend and watch it on the front end, but I don't know how to pass values the other way without an e/server block (which is basically the network).
I'll have to think about it.#2023-07-2921:47Dustin Getzyou need an atom on the frontend in local scope to mediate the bridge#2023-07-2921:47Dustin Getzcallback swaps local atom#2023-07-2921:48Dustin Getzwatch atom to get electric reactive value#2023-07-2921:48Dustin Getz now you can use e/server on that value #2023-07-2921:49Reut SharabaniI can watch client atoms? I thought it only works the other way around. Let me mess with it for a bit see if I understand#2023-07-2921:50Dustin Getz(when-some [x (e/watch !mediator)] (e/server (println x)))#2023-07-2921:50Dustin Getzthat’s a “callback”#2023-07-2921:51Dustin Getzmaybe you need to reset atom to nil after it depends on your use case#2023-07-2921:51Reut SharabaniI understand the other stuff, just the fact you can e/watch on the client was missing to me#2023-07-2921:51Dustin Getzit is probably worth reading about the m/observe approach too even if you don’t use it, because you’ll end up there eventually it just might take a couple tries for it to “click”#2023-07-2921:52Reut Sharabaniyeah it feels kind of familiar jargon-wise because I used to use rx-java a lot (even contributed an zip operator a long time ago)#2023-07-2921:52Reut Sharabanibut I don't know if it's actually the same#2023-07-2921:53Reut Sharabanione day I'll have a big enough commitment to read about missionary 🙂#2023-07-2921:54Dustin Getzoh ok great yeah missionary is like that#2023-07-2921:54Dustin Getzit’s learned best by example#2023-07-2922:02Reut Sharabaniatom works btw! I was just missing that I can (e/client (e/watch some-atom)) and not just (e/server (e/watch some-atom))#2023-07-2922:27braai engineerIf cc/let implies a do, and do is parallel in Electric, what is the execution order of let in Electric? Do the let-clauses execute sequentially or in order of topological dependence if a subsequent clause depends on the one above, or is there an m/sp in there somewhere?#2023-07-2922:40Dustin Getzthe let bindings evaluate sequentially when the let boots, the do is implicit only in the let body#2023-07-2922:41Dustin Getzany unused let binding will still be evaluated for effect (same as clojure)#2023-07-2922:43Dustin Getzeven when we say do is parallel, it still is touched left to right on first render#2023-07-3011:11braai engineerthx#2023-07-3011:14Dustin GetzYour question is great – let may indeed match semantics of do in terms of what happens if an exception is thrown (e.g. pending) in an intermediate let binding – i think the rest of the let bindings will still evaluate#2023-07-3011:14braai engineerIf one of the let bindings changes, do all the let-clauses evaluate in lexical order?#2023-07-3011:15braai engineere.g.
(let [a X
_ (prn "thing")
b Y ;; e.g. if Y changes
c Z]
(+ a b c))#2023-07-3011:15Dustin Getzreactive updates are fine-grained, only the expression whose input changed will rerun#2023-07-3011:15Dustin Getzso put the prn last, if Y changes the prn should not "reboot" because it's argument is constant#2023-07-3011:16Dustin Getz@U09FL65DK is this correct have I made any mistake?#2023-07-3011:16braai engineerso prn won't rerun unless it touches Y?#2023-07-3011:16Dustin Getzprn has invariable arguments (the string literal is constant), wrt updates the topographical ordering is irrelevant#2023-07-3011:19braai engineerso my understanding of Electric today is that the prn will only run on boot (in order of let-clauses) unless prn touches a binding that changed, in which case it will rerun in...unspecified order?#2023-07-3011:19braai engineere.g. if there are two prns that touch Y#2023-07-3011:20braai engineer(let [a X
b Y ;; e.g. if Y changes
_ (prn :first Y) ;; are order of prn effects defined?
_ (prn :second Y)
c Z]
(+ a b c))#2023-07-3011:23Dustin Getzhere, ummm i think when Y updates the two prns will be touched in lexical order (left to right) though technically this may be undefined behavior today#2023-07-3011:24Dustin Getzthey will absolutely be touched in the same reactive propagation frame, you'll see neither or both, never inconsistent#2023-07-3011:25braai engineerasking because you can imagine clojure peeps expect let to be ordered, so if two effects depend on Y (and only Y), you might expect both effects to run in the order they are defined.
I don't have a concrete case, but you could imagine transacting something and then blocking on the tx.#2023-07-3011:26Dustin Getzi think we currently satisfy that#2023-07-3011:27Dustin Getznote it's complicated, if instead of prn we have more Electric components (deeper DAG), inside that DAG may be more latency#2023-07-3107:42xificurCordering is largely UB (undefined behavior) and subject to change. Today those 2 prns will run in program order. Order of them re-running when Y changes is also UB. If you have a specific case where ordering is necessary we can take a look#2023-07-3111:45Dustin Getz@U09FL65DK i think we intended to lock this down (as the impl currently does the desired thing despite not being formally guaranteed), right? Subject to any larger design changes the future might hold#2023-07-3111:51xificurCwe discussed ordering guarantees but that issue is on the backlog. So yes, I don't see any changes happening to that anytime soon#2023-07-3112:03braai engineerNo specific cases right now - just trying to understand Electric's behaviour. Anytime I need to think about order I just put it in a cc/defn outside e-fn.#2023-07-3011:13braai engineerThis component that shows waiting time on a ticket at second-granularity ran for a while and then caused a 1011 Server process crash:
(defn wait-time->color
"turn anything older than 2 seconds red"
[seconds]
(if (> seconds 2) "red" "blue"))
(defn Date->seconds [date]
(int (/ (.getTime date) 1000)))
(e/defn Card [tid]
(e/server
(let [ticket (d/entity db tid)
timestamp (:ticket/timestamp ticket)
now-seconds (int e/system-time-secs)
waiting-time-s (- now-seconds (Date->seconds timestamp))]
(e/client
(dom/div
(dom/props {:class "card" :style {:background-color (wait-time->color waiting-time-s)}})
(dom/text "Card"
" ⏰" waiting-time-s "s"))))))
~Haven't reproduced yet~, but am I using e/system-time-secs correctly? I see that e/system-time-secs is still millisecond "granularity".#2023-07-3011:16braai engineerreproduced. it crashed again after a few minutes of counting. Electric "v2-alpha-349-ge9996713"#2023-07-3012:06braai engineerException:
ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure
java.io.InterruptedIOException: null
at org.eclipse.jetty.util.FutureCallback.block(FutureCallback.java:153)
at org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint.sendBlocking(JettyWebSocketRemoteEndpoint.java:224)
at org.eclipse.jetty.websocket.common.JettyWebSocketRemoteEndpoint.sendPing(JettyWebSocketRemoteEndpoint.java:169)
at hyperfiddle.electric_jetty_adapter$make_heartbeat$cr47371_block_2__47377.invoke(electric_jetty_adapter.clj:20)
at cloroutine.impl$coroutine$fn__4431.invoke(impl.cljc:60)
at missionary.impl.Sequential.step(Sequential.java:88)
at missionary.impl.Sequential$1.invoke(Sequential.java:112)
at missionary.impl.Sleep.trigger(Sleep.java:63)
at missionary.impl.Sleep$Scheduler.run(Sleep.java:36)
Caused by: java.lang.InterruptedException: null
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1133)
at java.base/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:276)
at org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:125)
at org.eclipse.jetty.util.FutureCallback.block(FutureCallback.java:147)
... 8 common frames omitted
DEBUG hyperfiddle.electric-jetty-adapter: Client disconnected for an unexpected reason. {:status 1011, :reason Server process crash}#2023-07-3012:19braai engineer@dustingetz I had to move e/system-time-secs to the client and it hasn't crashed yet, but not ideal.#2023-07-3012:27Dustin Getzthis is unexpected let me review closely#2023-07-3012:28braai engineerNo other logs, but it's possible I have old log config. This is my logback.xml:
<!-- Jetty logger config. See and
ask ChatGPT. `scan` and `scanPeriod` enable hot reload of logger config, leave on! -->
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%highlight(%-5level) %logger: %msg%n</pattern>
</encoder>
</appender>
<root level="WARN">
<appender-ref ref="STDOUT" />
</root>
<logger name="hyperfiddle" level="DEBUG" additivity="false"><appender-ref ref="STDOUT" /></logger>
<!-- <logger name="hyperfiddle.electric.impl.io" level="INFO" additivity="false"><appender-ref ref="STDOUT" /></logger> -->
<logger name="app" level="DEBUG" additivity="false"><appender-ref ref="STDOUT" /></logger>
</configuration>#2023-07-3012:35Dustin GetzI think this is a heartbeat issue related to something we fixed this week in master#2023-07-3012:36Dustin GetzDid you blur the tab, such that the tab was not in the foreground while you looked at another tab?#2023-07-3012:38braai engineerlet me double check. it is in own window in foreground so should have "focus" but maybe I blurred it#2023-07-3012:39Dustin Getzyeah i suspect the issue is not actually connected to e/system-time-ms at all#2023-07-3012:39Dustin Getzthe sleep in the stack trace is related to the websocket heartbeat to keep the connection alive#2023-07-3012:40braai engineeram now running two windows: one focused, one blurred. let's see if blurred breaks#2023-07-3012:41Dustin GetzBTW, be aware that when you stream a clock from the server to the client, that can consume a lot of bandwidth. Here the clock is truncated to 1hz so that should be a lot better than streaming every tick - which can like saturate the wire in theory#2023-07-3012:42braai engineeryup, the blurred window died. focused still running#2023-07-3012:44braai engineercan Electric reconnect & resync yet? or force reload?#2023-07-3012:46Dustin Getzi think this is just a bug , we will try to release the fix this week#2023-07-3012:46Dustin Getzresync is blocked on delivery of the next electric version first#2023-07-3012:47braai engineerk so I did the same test but with client side time, and it does not crash#2023-07-3012:49Dustin Getzthanks, that is interesting#2023-07-3011:24braai engineerI am building some cool shit in Electric. Something simple but big is coming soon!
I have been waiting for Electric my whole "career". I used to 'mock' stuff in Reagent in DataScript and wasted sooo much time trying to get data sync to server working coz I'm not as smart as @robert-stuttaford, but now I can just outsource my IQ to @dustingetz &@leonoel.#2023-07-3011:28braai engineeralso @robert-stuttaford how are you not in here?? you need to be tracking Electric.#2023-07-3011:28Dustin Getzany teasers about your thing? 🙂#2023-07-3107:14robert-stuttafordglad to hear you're digging it @U051SPP9Z 😄#2023-07-3107:14robert-stuttafordand thank you for the kind words! 😊#2023-07-3012:21braai engineerWhy does -get-system-time-m definition have this signature in args: [& [_]]?
(cc/defn -get-system-time-ms [& [_]] #?(:clj (System/currentTimeMillis) :cljs (js/Date.now)))
#2023-07-3012:30Dustin Getz(hyperfiddle.electric/def system-time-ms "ms since 1970 Jan 1"
(if (= "visible" dom-visibility-state)
(new (m/sample -get-system-time-ms <clock))
(throw (Pending.)))) ; tab is hidden, no clock. (This guards NPEs in userland)
#2023-07-3012:31Dustin Getzm/sample calls -get-system-time-ms once per clock tick, m/sample passes the current time, -get-system-time-ms ignores it because it is sampling the system clock by side effect#2023-07-3012:32Dustin Getz<clock is the efficient requestAnimationFrame clock (ticker, really) running at the browser animation rate#2023-07-3012:35braai engineerDoes it need to pass the current time to...force an effect? Or just happens to be implemented that way?#2023-07-3012:50Dustin Getzno, this is just type tetris w/ m/sample#2023-07-3013:19braai engineerWhy am I getting illegal invocation for this?
(dom/on "click" (e/fn [e]
(e/client
(let [new-name (js/prompt "Rename" (or (:entity/name entity) ""))]
(if-not (string/blank? new-name)
(e/server
;; todo auth.
@(d/transact conn [[:db/add eid :entity/name new-name]])
nil))))))#2023-07-3013:19Dustin Getzshow error please#2023-07-3013:20braai engineer#2023-07-3013:22Dustin Getzi think that error came from Clojure not Electric#2023-07-3013:22braai engineerThe JS prompt does not even show#2023-07-3013:22Dustin Getzdoes it take a window reference as the first parameter?#2023-07-3013:23Dustin Getz(js/console.log) is (.log js/console)#2023-07-3013:23braai engineerno, but prompt lives on both#2023-07-3013:23Dustin Getznot sure what (js/prompt ) transpiles to#2023-07-3013:23braai engineerjs/window.prompt also fails#2023-07-3013:24Dustin Getzsame error?#2023-07-3013:24braai engineeryes#2023-07-3013:24Dustin Getzi would get this working outside of electric first if we can confirm this is the line that is failing#2023-07-3013:25braai engineerworks if I move it out into own fn#2023-07-3013:25braai engineer(defn prompt-rename [qid name]
#?(:cljs (js/prompt "Rename queue" name)))
#2023-07-3013:25braai engineer^ works#2023-07-3013:25Dustin Getzhm ok could be a compiler bug#2023-07-3013:26Dustin Getzyou can put the reader conditional on the outside btw#2023-07-3013:26braai engineeryeh#2023-07-3019:18Garrett HopperHad the exact same issue this morning with js/alert :thinking_face: (But not js/console.log which was strange)
It also worked when I wrapped it in its own defn that passed along the string. 🤷#2023-07-3019:18Dustin Getzlogged a ticket#2023-07-3107:47xificurCthis is fixed on master https://github.com/hyperfiddle/electric/commit/6835905a744ecbece019e0ddfc43e68220e009a5 , not yet released#2023-07-3019:16Garrett HopperIf I have multiple (side-effecting) functions that I want to run when a single reactive variable changes, is there any way to do so while keeping the functions inline. (I could create an external defn (non-reactive) which would then be called with the arguments, however it'd be nice to inline the calls)
E.g.
(defn do-both-things [x y]
(do-thing x)
(do-other-thing-also y))
(e/defn Example []
(let [!x (atom nil) x (e/watch x)]
(do-both-things x y)))
;; VS
(e/defn Example []
(let [!x (atom nil) x (e/watch x)]
(do-when [x] ;; <--- How?
(do-thing x)
(do-other-thing-also 1234))))#2023-07-3019:21Dustin Getzyou cannot do this today without wrapping it in a clojure lambda. What is the use case?#2023-07-3019:22Dustin Getz((fn [] (f x) (g))) can be inlined in Electric regions, this is what such a macro would do if we supplied it, which we are disinclined to do#2023-07-3019:23Dustin Getzthat will rebuild the clojure lambda when x changes and thus call it again, and the clojure lambda has clojure semantics#2023-07-3019:29Garrett HopperHmm, that IIFE isn't a bad idea.
I don't know that there's a strong use case for it. (In most situations it'd be more desirable to create a external function anyways.)
The use case I specifically ran into was some canvas context manipulation logic which requires multiple calls to set up the imperative context prior to the final call which actually depends on the reactive variable.#2023-07-3019:32Dustin GetzYeah when dealing with foreign interfaces this happens a lot. We intend to provide better idioms here. One major issue with Electric today is that missionary m/ap macro (which is probably the right way to sequence imperative logic), its macroexpansion has a teeny tiny array mutation in Cloroutine way down the stack which is incompatible with Electric's evaluation model#2023-07-3019:32Dustin Getzwhich we need to fix#2023-07-3019:34Garrett HopperInteresting :thinking_face:
I kinda of expected there'd be some sort of missionary specific implementation that would do this.
May have to dig into the m/ap expansion to understand what you're saying about it being incompatible with Electric at the moment.#2023-07-3019:35Dustin Getzm/ap macroexpands to cloroutine which macroexpands to low level array stuff for performance, i believe#2023-07-3019:35Dustin Getzthat assumption (of secret internal-only mutation being harmless) suddenly being violated when that macroexpansion occurs inside electric evaluation context and is suddenly reactive#2023-07-3019:37Garrett HopperElectric functions are themselves just being compiled to missionary flows (?), right?#2023-07-3019:37Garrett HopperStill trying to connect the dots between my (limited) understanding of missionary and how Electric programs are compiled into it.#2023-07-3019:57Garrett HopperErr, I'm realizing my mental model was off.
I thought any missionary flow could be embedded in Electric by calling new on it.
I didn't realize e/def was needed on the m/ap definition.
(e/def example-flow (m/ap "a" "b"))
(e/defn Example []
(dom/div (dom/text (new example-flow))))
#2023-07-3020:03Dustin Getzit’s not, any missionary flow can be “signal awaited” with new, subject to the constraint that the flow has an initial value#2023-07-3020:03Dustin Getzm/ap is not supported in electric code however due to the cloroutine issue#2023-07-3020:03Dustin Getzyou’ll need to wrap in clojure lambda#2023-08-0115:15Garrett HopperI once again find myself wishing for a React.useEffect style macro that would allow an implicit do to be re-run whenever an input array of reactive vars are changed.
(I am open to the possibility of it being a bad idea; I haven't thought about it very deeply yet.)
It would be slightly different from just wrapping the logic in a defn, as the body would have access to context variables in the closure without having to explicitly pass them to the function.#2023-08-0115:21Dustin Getzm/observe is react.useEffect except with automatically correct dependency tracking#2023-08-0115:22Dustin Getzreally the only viable path forward is to make the dependencies explicit in the dag with an IIFE#2023-08-0115:22Dustin Getzthe root cause of the pain is dropping out of the immutable information model#2023-08-0115:22Garrett Hopper👍 Thank you 🙂#2023-08-0115:25Dustin Getzoh, note that using m/observe to sequence here involves a clojure lambda, so it accomplishes what the IIFE accomplishes except with more machinery#2023-08-0115:27xificurC> the body would have access to context variables in the closure without having to explicitly pass them to the function
are you sure this doesn't work? E.g. this works
(e/defn Foo []
(let [x 1]
(#(inc x))))#2023-08-0115:32Garrett HopperThe IIFE would, yes
TBH, I'm not sure exactly what I thought would be solved by a macro when I wrote that.
Maybe the ability for it to not recreate the function every time a react var changes?
I think I'm satisfied that there's nothing clear to be gained here at the moment.#2023-08-0115:51Dustin GetzI would prefer to let the compiler optimize at that layer, we don't optimize much yet but optimizations like avoiding closure allocation can probably be done mechanically#2023-07-3102:08joshchoHow do we pass a client-side atom to a server-side e/defn? I am getting Unserializable reference transfer: cljs.core/Atom [object Object] .#2023-07-3102:45joshchoEncountered here, making an input that syncs with server every n seconds:
#?(:clj (defonce !src-text (atom nil)))
(e/defn BackpressuredInput
[!text !src-text interval]
(e/server
(let [src-text (e/watch !src-text)]
(e/client
(let [text (e/watch !text)]
(when (and (not= src-text text)
(= 0 (int (mod e/system-time-secs interval))))
(e/server (reset! !src-text text)))
(ui/input
text (e/fn [v]
(reset! !text v))))))))
(e/defn ExperimentApp []
(dom/text (e/server (e/watch !src-text)))
(dom/div
(let [!text (atom "Hello")]
(e/server
(BackpressuredInput. !text !src-text 5)))))#2023-07-3108:06xificurCatoms don't serialize over the wire. It'd also be scary if they would, since any client could locally alter it!
In your example you could push !text into BackpressuredInput completely#2023-07-3109:42joshchoThat makes sense. Are you suggesting something like this?
(e/defn BackpressuredInput
[val !src-text interval]
(e/server
(let [src-text (e/watch !src-text)]
(e/client
(let [!text (atom val)
text (e/watch !text)]
(when (and (not= src-text text)
(= 0 (int (mod e/system-time-secs interval))))
(e/server (reset! !src-text text)))
(ui/input
text (e/fn [v]
(reset! !text v))))))))
(e/defn ExperimentApp []
(e/server
(binding [db (e/watch !conn)]
(e/client
(dom/text (e/server (e/watch !src-text)))
(dom/div
(dom/props {:class "m-10"})
(e/server
(BackpressuredInput. "Hello" !src-text 5)))))))
It would be ideal if I had access to the !text atom outside of it.#2023-07-3111:50xificurCcorrect.
In the future your code will work as-is. Today all arguments to an e/fn are transferred, which causes the above issue. This is an issue that requires some lower-level fixes first. Today you can bypass the issue by changing BackPressuredInput to a macro. The macro expands inline, therefore it's not subject to unwanted transfers#2023-07-3112:09joshchoWhat do you mean by all arguments are transferred? I want to get a better model of how Electric works.#2023-07-3112:22xificurCIn a normal clojure call (foo bar (baz quux)) the arguments are evaluated before calling foo. Electric copies this behavior and the compiler ensures all arguments are evaluated. So (BackpressuredInput. !text !src-text 5), which is a server call, evaluates its arguments on the server immediately, forcing the transfer of !text, since it's a client-side value.
Turns out copying clojure's evaluation model for electric is unsound for the issue you just ran into - you are passing in a client-side value and you're only using it on the client. So transferring it to the server is pointless and harmful. This is evident from the fact that refactoring the e/defn to a macro solves the problem you're facing.
In the future the compiler will be smarter and skip that unnecessary transfer since it breaks composition.
Was that helpful / understandable? 🙂#2023-07-3113:19Dustin GetzSummary : currently, when you invoke an electric fn like (e/client (new Foo-query (e/server db) y)) , the new "happens" on the client, which means all arguments are transferred to the client, including db. Now consider (e/defn Foo-query [db y] (e/client (dom/div (dom/text (pr-str (e/server (query ... db)))))) , db is not actually used on the client. Unfortunately, today, db will (1) round-trip for no reason (2) become nil in the process due to unserializable reference transfer. That's bad, you should be able to pass to an electric function arguments of both server color and client color. It's getting fixed soon.#2023-07-3113:22Dustin Getztemporary workarounds:
• pass by dynamic scope
• call new on the server, which tends to make this problem go away due to the natural server-to-client dataflow of web apps
• use defmacro to route around "new" entirely, and control where db is spliced#2023-07-3113:53joshchoThank you @U09FL65DK and @U09K620SG! Makes sense; I will use a workaround and wait for new features. Electric is very cool.#2023-07-3113:45frozenlockGreetings. Finally getting my toes wet after reading about electricfor the last few months/years. I have a small DB/storage wrappers and I'd like to be able to use it with electric. I quickly glanced at missionary, but all of this is pretty new and I have a hard time figuring out how I should approach this.
Let's say I have full control (can redefine) my storage API (`db/assoc-at!`, db/get-at ...), how should I approach this to make it compatible? I'm pretty sure I could re-read everything every time a modification occurs, but that seems wasteful. What about targeted reaction? For example, only reprocess this query if a particular document was modified.#2023-07-3114:26Dustin Getzhow is your API used outside of electric? the problem of knowing when to refresh a query is introduced by Electric's streaming model yes?#2023-07-3114:40frozenlockFor now, really simple HTTP stuff.
Go to URL, display the document. If the user wants an updated view, he needs to reload the page.
So yes, using Electric introduces a new reactive component which I'm trying to figure out.#2023-07-3114:50frozenlockOhhh I found the tutorials 😅#2023-07-3114:50frozenlockhttps://github.com/leonoel/missionary/wiki
This will most likely contain the answers I'm seeking.#2023-07-3114:51Dustin GetzPerhaps the right answer for you is to refresh queries when the user navigates#2023-07-3114:51Dustin GetzThus using the library with the same data access patterns it was designed for, while also getting the benefit of electric#2023-07-3114:53frozenlockRight. But I'm the one designing the DB wrapper library, so if I can also add some reactivity already that'd be great. Especially if it requires modifying its API.#2023-07-3114:53frozenlockI'll peruse the tutorials and come back if I'm confused simple_smile#2023-07-3114:53Dustin Getzok#2023-07-3114:55Dustin Getzif the data access library can invoke a callback to push out updated data efficiently, electric can trivially integrate it. The problem is that the data layer needs to determine when a subscribed query has updated. Otherwise, we can resort to polling approaches (implemented with Missionary)#2023-07-3115:34Garrett HopperAre #js literals expected to work in cljs? (inside an electric function)#2023-07-3115:35Garrett HopperCurrently working around with clj->js, but it's not quite the same.#2023-07-3115:36Dustin GetzNo they aren't implemented yet sorry, just confirmed, maybe we can sneak it in soon#2023-07-3115:36Dustin Getzi escalated the ticket#2023-07-3115:37Garrett Hopper👍 No big deal; thank you 🙂#2023-07-3116:44Joseph WinstonWarning - New User Question
Can Electric only use ClojureScript? Then I don't have to worry about two different package managers, build systems, etc.#2023-07-3116:44Dustin GetzUnfortunately not today#2023-07-3116:45Joseph WinstonThanks for the quick reply.#2023-07-3117:03Joseph WinstonI'd really welcome using ClojureScript/Node.js for the backend.
Is it worthwhile at this point in Electric's life to even think about this?#2023-07-3117:06Dustin Getzi think our language tests pass on nodejs but the compiler is still JVM, there are almost certainly major gaps that are unfinished and that work would not happen this year #2023-07-3117:06Dustin Getzsorry#2023-07-3117:07Dustin Getz(unless a corporate sponsor funds the work)#2023-07-3117:43Joseph WinstonI wish that I had someone to pay... But at this point in time, I do not.#2023-07-3117:43Dustin GetzHa, yes of course 🙂#2023-07-3121:09frozenlockcontrib.str appears to be required only for contrib.str/empty->nil. Couldn't the built-in not-empty be used instead? Or am I missing something?
https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L2#2023-07-3121:15Dustin Getzi may not have known about not-empty, are you certain they are the same?#2023-07-3121:16frozenlockIt's a more general function (coll instead of strings), but it looks like it would do the job there.#2023-07-3121:18frozenlock> i may not have known about not-empty
I myself only noticed it a few months back.#2023-07-3122:02Dustin Getzi’ll merge a PR if you want the contributor badge, otherwise will remove it next time i see it#2023-07-3122:02Dustin Getzty#2023-08-0100:04frozenlockSure, I'll do it tomorrow if that's okay.
I saw it in more than one example, so I'll try do look around for other instances.#2023-08-0101:02Dustin Getzsure no worries, i’ll get to it eventually i just wanted to make sure you get the badge if you want it :)#2023-08-0121:22frozenlockI tried running the repo locally, but it fails when trying to fetch the datomic dependency (401 unauthorized).
This is becoming a little too involved just for the small update 😅
I leave it up to you.#2023-07-3121:17frozenlocke/fn and e/defn don't return/define usual functions. When used alone, they throw Invalid e/server in Clojure code block (use from Electric code only). Is the REPL part of the development process when building a webpage with electric?#2023-07-3121:52Dustin Getzhttps://www.reddit.com/r/Clojure/comments/158c4lk/comment/jtddii1/#2023-07-3121:53Dustin Getzit depends what you’re doing; if you’re building web ui i don’t think the repl will help you any more than it does with reagent#2023-07-3121:54Dustin Getzthe reddit reply explains how to do it#2023-07-3121:57frozenlockI'm finding the REPL quite useful with reagent.. there's always some bug that require a way to intercept the data and experiment with it. 😅
Thank you for the link! In this case it will be useful to play around the fns/reactions#2023-07-3121:59Dustin Getz how can the repl help you with reagent when react.js controls evaluation?#2023-07-3122:07frozenlockReagent controls the rendering, but it's easy to def a cursor and inspect it in the REPL. Functions generating hiccup can also be called directly.
I'm still new with Electric, so it's not immediately obvious to me where I can anchor myself when I need to inspect something, or make sure a function is returning what I'm expecting.
It's probably just a habit thing; once I'll know more it will likely be obvious.#2023-07-3122:09Dustin Getzoh ok, deep def works here too#2023-07-3122:09Dustin Getzif that’s what you mean#2023-07-3121:34Garrett Hopper~I see ~#2023-07-3121:38Garrett Hopperhyperfiddle.electric-svg... :man-facepalming:#2023-07-3121:51Garrett Hopperhttps://hyperfiddle.github.io/#/page/64bfd712-3a3b-4c11-b8d6-0a5a0f63e098
> the server is compiled by the clojurescript build Electric -> DAG IR, and the server DAG value (IR, think bytecode) is embedded into the javascript artifact. The browser client will bootstrap the server by sending up the DAG value to the server in the first frame of the websocket, and then the server interprets the DAG IR at runtime.
> (Yes, that means the Electric frontend is compiled all the way to JS but the Electric backend is only partially compiled and then interpreted)
> There may be minor errors in this explanation but this is the gist of it.
> We have a branch that bakes the server DAG into the java artifact rather than bootstrapping from the client, i forget our exact plans for this but there will be changes this summer
I see the IR being sent from the client when it first connects to the WS.
I'm unclear what exactly this contains though. In theory, could it be modified by the client to cause undesirable evaluations on the server? (At least until the server DAG can eventually be embedded directly in the JAR?)#2023-07-3121:57Dustin Getzhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1689922421825169#2023-07-3122:04Garrett HopperThank you! 🙂#2023-07-3122:15braai engineerHmm, gotcha: (d/touch (d/entity db eid)) does not update reactively but (d/pull db '[*] eid) does, probably because I misunderstand d/touch, but unexpected.#2023-07-3122:16Dustin Getzhmm interesting, first problem is d/entity has broken equality and they should fix it#2023-07-3122:17Dustin Getzi think that is the problem here#2023-07-3122:18Dustin Getzthere is a ticket on http://ask.datomic.com#2023-07-3122:20braai engineerupvoted: https://ask.datomic.com/index.php/859/equality-on-d-entity-ignores-db?show=859#q859#2023-07-3122:23braai engineerWhy is d/touch not running in my snippet tho, coz db changed? Is Electric reaching into that S-expr?#2023-07-3122:23braai engineerI always thought d/touch was equivalent to d/pull *#2023-07-3122:48Dustin Getzit gets work skipped because the fresh entity is = to the old one#2023-07-3122:48Dustin Getziiuc#2023-08-0107:12xificurCcorrect. In (foo bar) foo will re-run when (not= new-bar old-bar). In your case datomic's old and new entity are considered clojure.core/=, so d/touch reuses the previous value instead of rerunning#2023-08-0100:08frozenlockIn the chat example, is there a particular reason why each msg is sent individually instead of using msgs directly in the client?
(e/server
(e/for-by identity [msg (reverse msgs)] ; chat renders bottom up
(e/client
(dom/li (dom/style {:visibility (if (nil? msg)
"hidden" "visible")})
(dom/text msg)))))
Was it to show the interaction of nested client/server?#2023-08-0101:01Dustin Getzi don’t understand the question?#2023-08-0101:51frozenlocke/for-by is under server, but couldn't the whole thing be done on the client?
Something like this:
(e/defn Chat []
(e/client
(try
(dom/ul
(e/for-by identity [msg (reverse msgs)] ; chat renders bottom up
(dom/li (dom/style {:visibility (if (nil? msg)
"hidden" "visible")})
(dom/text msg))))
(dom/input
(dom/props {:placeholder "Type a message"})
(dom/on "keydown" (e/fn [e]
(when (= "Enter" (.-key e))
(when-some [v (empty->nil (.. e -target -value))]
(e/server (swap! !msgs #(cons v (take 9 %))))
(set! (.-value dom/node) ""))))))
(catch Pending e
(dom/style {:background-color "yellow"})))))
msgs is already defined on line 9. I don't quite get why this needs another server interaction for the for-by loop.#2023-08-0107:24xificurCThis is a common pattern. Since msgs is a server value you want to diff it on the server. e/for-by diffs and maintains the list incrementally. Doing the whole thing on the client will compile but you'll be re-sending the whole list of msgs on every change#2023-08-0111:12frozenlockI see. And will all the li elements be updated because their value changed, or will it only add the new msg and shift the others positions in the list?#2023-08-0111:23xificurCgood question. Updates are fine grained, so if a new value comes in it's the only one that's going to create#2023-08-0111:24frozenlockSo the sever will only send the info for the new msg and the old one that was discarded?#2023-08-0111:39xificurCcorrect#2023-08-0111:40frozenlockImpressive!#2023-08-0100:09frozenlockFor reference, I'm talking about this example: https://electric.hyperfiddle.net/user.demo-chat!Chat#2023-08-0102:14joshchoI have code to hide the element until loaded like this:
(dom/div
(try
(dom/text (e/server (get-text)))
(catch Pending e
(dom/props {:class "hidden"}))))
I have two questions:
1. How do I instead show a different element until loaded, like a loading symbol?
2. I just want this to only wait for load only on mount. Is that possible? Essentially, the try/catch block would only exist when it mounts.#2023-08-0107:49xificurCTo mount A xor B you need a conditional, e.g.
(if-some [txt (try (e/server (get-text)) (catch Pending _))]
(dom/div ...)
(Spinner.))
If (get-text) returns Pending only on mount then you're done. Otherwise a
quick&dirty way would be to wrap the above code with an atom to differentiate
between first and subsequent Pendings
(let [!mounting (atom true)]
...)
#2023-08-0111:43Dustin Getz@U6D1FU8F2 i recommend not trying to battle with Pending if possible, just let the page stream in for now#2023-08-0111:44Dustin GetzI say this because there are some aspects of Pending that we are internally a bit confused about (e.g. does Pending always match what you'd want to be "Loading" or is Pending sometimes seen at times that you would not consider to be Loading – we aren't quite sure)#2023-08-0111:46Dustin GetzIf you need to do this it's fine, just stay in touch so we can understand your use case#2023-08-0114:27henrikI’ve run into cases where I’ve found it a bit hard to get things to happen in a desired order wrt rendering. For example, in a list, I’ve found that it’s hard to fix the race condition of items being added to the top of the list vs. scrolling to the currently highlighted item.
If the added items pushes the highlighted item below the fold, scrolling to the highlighted item will pretty much never happen, since the data that triggers the scrolling event (i.e.: the list content changed) is available before the DOM elements are mounted on screen, rather than happening in the “same transaction” so to speak.
With Pending, it seems that Pending -> Data available -> DOM is rendered happens in three discrete steps as well, so it’s hard to go directly from spinner to the desired DOM content without passing an undesired empty DOM state. This specific case I’m thinking of goes via for-by though, so maybe it’s related to the eagerness you mentioned previously @U09K620SG.
I’m thinking it might be possible to fix both by sending back dom/node as a signal that everything is mounted and visible, but I haven’t tested yet. If it works, the second step will be to figure out how to do it so that it doesn’t result in a mess of spaghetti.#2023-08-0114:29Dustin GetzIs this demonstrated in the project repo you sent me?#2023-08-0114:38henrikNot quite, I removed the code when I couldn’t get it to work. I’ll add it back and ping you.#2023-08-0114:43Dustin GetzPlease do, thanks. We are planning some changes here so I do not recommend you add hacks#2023-08-0114:44Dustin Getzand the e/for latency issue is involved#2023-08-0114:44Dustin Getztransactional dom writes is being considered#2023-08-0114:48henrikThe DOM is very untransactional in its natural state. A lot of effort has gone into React to hide this fact, AFAIK.#2023-08-0114:49Dustin Getzwe understand#2023-08-0115:02henrikYeah, not to say that the solution that React delivers isn’t without its problems either. They chose to set the granularity at the component level, so it’ll provide some transactionality at the level of the component.
The fact that Electric is more granular than that is a major strength in most cases, I would say.#2023-08-0115:03Dustin Getzreact is also limited by the reconcilier which makes assumptions#2023-08-0115:04Dustin Getzi'm not the engineer who did the research so far so i dont want to comment too much, but we did do a discovery spike to see what other frameworks are doing#2023-08-0116:10joshchoThank you @U09K620SG. A lot of drive in this project makes me trust it more.#2023-08-0116:13joshchoIn terms of lists from e/watch, how does Electric handle rerendering? Suppose the watched variable goes from (a b c) to (a d b c e b). If there is a for loop over the variable, how does render/rerender happen?#2023-08-0116:13joshchoI guess it's related to https://clojurians.slack.com/archives/C7Q9GSHFV/p1690889713294449?#2023-08-0116:30Dustin Getzyes, search for e/for-by#2023-08-0116:30Dustin Getzalso review https://electric.hyperfiddle.net/user.demo-system-properties!SystemProperties#2023-08-0116:31Dustin Getzalso https://clojureverse.org/t/electric-clojure-a-signals-dsl-for-fullstack-web-ui/9788/32?u=dustingetz#2023-08-0116:43joshchoReally interesting. So a reactive database like https://github.com/mhuebert/re-db is not strictly necessary? Electric just takes care of the diffs in db watch naturally, and stops propagation when a higher node in the s-expr tree is equivalent.#2023-08-0116:44joshchoI found myself being confused (then pleasantly surprised) when my console logs weren't being printed when I thought the "component" should re-render. It takes some time to get used to the fine-grained nature of things, and how things sort of "just work" at the compiler level.#2023-08-0116:50joshchoBut I guess re-db can be useful since with something like (e/server (some-query db)) will re-run every time !conn updates, whereas a reactive database can prevent that from happening. But at least upstream, this won't trigger an update if (some-query db) hasn't changed.#2023-08-0119:24Dustin GetzYeah, it's great to have reactivity/streaming inside the data layer, and if you do have a streaming data layer, Electric can speak that natively#2023-08-0119:25Dustin GetzUnfortunately, knowing when to refresh your queries (for a data layer that is not streaming) is a hard problem, and it's not one that Electric solves out of the box#2023-08-0119:28Dustin GetzMost databases, e.g. Postgres and Datomic, provide a notification API so that you can understand what changed and use that information to know which queries to recheck – for many apps a simple heuristic will do. We can also help you throttle various queries as not all of your app needs to be realtime – in a scaled situation likely only certain restricted components, like a chat component, would be realtime, and that would be backed by a message queue or document store that offers efficient subscription#2023-08-0200:05joshchoThanks @U09K620SG, makes a lot of sense.#2023-08-0107:37braai engineerMultiple Electric builds per project - possible? E.g. Shadow-cljs can have different builds defined for when you need a lightweight release for customers and a heavy build for admins / managers. Multiple "user.cljs"? Or do I need multiple Java runtimes for this?#2023-08-0109:05Geoffrey GaillardNo need for multiple java runtimes. It should work exactly how you described it.#2023-08-0111:28frozenlockThere's some black magic happening in https://electric.hyperfiddle.net/user.tutorial-backpressure!Backpressure
How does it work? (int s) is compiled into a flow?
> • Electric is a Clojure to https://github.com/leonoel/missionary compiler; it compiles lifts each Clojure form into a Missionary continuous flow.
> #2023-08-0111:42xificurCyes, (int s) compiles roughly to (m/latest int s)#2023-08-0113:04Dustin GetzHere is a pseudocode (for understanding) of the compiler transformation :
https://github.com/hyperfiddle/electric/blob/f74a488e442c870b6a5409f45f49fd8a8a972240/src-docs/user/electric/electric_compiler.clj#L29-L31#2023-08-0113:05Dustin GetzThat pseudo-transformation excludes the client/server network management#2023-08-0113:05Dustin GetzThe real transformation is this: https://github.com/hyperfiddle/electric/blob/f74a488e442c870b6a5409f45f49fd8a8a972240/src-docs/user/electric/electric_compiler_internals.cljc#L27#2023-08-0113:06Dustin Getzthe Electric compiler uses a real Clojure/Script analyser to transform clojure/script syntax into "DAG bytecode" for a "DAG virtual machine"#2023-08-0113:07Dustin GetzElectric is basically the JVM but for network#2023-08-0113:08frozenlockFascinating#2023-08-0113:08Dustin GetzDon't be scared, this is why we waited 2.5 years to release anything publicly#2023-08-0113:08Dustin Getzit is robust#2023-08-0115:05Garrett HopperHow similar would you say this is to the SSA/VM macro logic of core.logic or https://github.com/leonoel/cloroutine/blob/master/src/cloroutine/impl.cljc?#2023-08-0115:22xificurCLeo wrote electric, missionary and cloroutine. With that said the electric VM is a VM of its own suited to solve the problems electric is meant to solve (a reactive network-transparent language)#2023-08-0115:36Garrett HopperThat's good to know. I knew he wrote Missionary and Cloroutine, but I hadn't realized he was directly involved with Electric as well. 🙂#2023-08-0115:46Dustin GetzIt's not SSA but cloroutine does implement a clojure/script analyzer and electric does too so they are similar in that aspect#2023-08-0115:47Dustin Getzfwiw cloroutine is not really on the stack in Electric, cloroutine is used to implement the missionary m/ap and m/cp syntax sugar macros but those are not essential to the Electric compiler transform#2023-08-0116:11frozenlockI weep for those not already familiar with Clojure and the cljs ecosystem.
Learning all of this at once would be challenging to say the least.
I've read the Missionary tutorial twice and it's still not sticking. 😒#2023-08-0116:13Garrett HopperFWIW, I don't think that new users in the future will need to know internal details like this.
Writing "Electric code" is straightforward. It's just for those that need to be convinced of the correctness of the internals that need to dig into these details. 🙂#2023-08-0116:16frozenlockHopefully. It's not yet clear to me yet what can be omitted.#2023-08-0116:28Dustin GetzMissionary docs need dramatic improvement, that's not on you#2023-08-0116:28Dustin GetzLeo is a walking computer science textbook#2023-08-0116:29Dustin GetzElectric also needs streamlining, we will do another pass this fall#2023-08-0111:33frozenlockHmm... I have some pretty involved algorithm in a .cljc file. If I use electric, does it mean it will automagically become reactive? :thinking_face:#2023-08-0111:35frozenlockSay it processes a coll of 1000 elements. Will it be able to re-process only the nth element if its incoming value changed?#2023-08-0111:41xificurCe/for-by can. Note that dropping a heavy algorithm as-is into electric will degrade its performance. But if you use e/for-by in the right place(s) it will be incremental#2023-08-0111:45frozenlockVery interesting#2023-08-0112:51Dustin Getzby "heavy alg" we mean, say, code designed for linear execution, like a database query engine#2023-08-0112:51Dustin GetzCompare to reactive/async execution which is IO bound#2023-08-0112:51Dustin Getzuse Clojure for linear code, use Electric for reactive code#2023-08-0112:57frozenlockI'm transforming timeseries data with a bunch of transducers.
Each transducer adds its own field to the the timeseries, potentially using values from a previous transducers.
For example:
{:t 1, :v 100}
{:t 2, :v 150}
;; After transducers 'plus5'
{:t 1, :v 100, :plus5 105}
{:t 2, :v 150, :plus5 155}
Would you say it's possible/desirable to leverage reactions?#2023-08-0113:00Dustin GetzI would use Missionary if your algorithm is concurrent#2023-08-0113:00Dustin GetzElectric includes network management which is not free#2023-08-0113:01Dustin GetzProcessing timeseries data sounds compute bound not IO bound#2023-08-0116:16joshcho@U09FL65DK Can you elaborate on e/for-by? What's the difference (rendering-wise) with e/for?#2023-08-0116:19frozenlockMy understanding is that for-by identifies the results with a 'key', similar to how it's done with components in Reagent.
Assuming (for [x coll] ...), with current value of x being {:id 1 :msg "hello"} :
In Reagent, we'd do something like this:
^{:key (:id x)} [:div (:msg x)]]#2023-08-0116:25Dustin Getz(e/for ...) is just (e/for-by identity ...)#2023-08-0116:25Dustin Getzhonestly we'll probably remove e/for , the only benefit is that the early tutorial code is slightly cuter (can delay the explanation)#2023-08-0116:39joshchoThat makes a lot of sense. In terms of internals, what happens if there are two elements with the same key?#2023-08-0207:11leonoel> In terms of internals, what happens if there are two elements with the same key?
The diff algorithm will choose the closest, according to position in sequence#2023-08-0112:04frozenlockhttps://electric.hyperfiddle.net/user.demo-chat-extended!ChatExtended
Line 50: do is used, but a non-last component appears in the final UI. ("Authenticated as")
Does it mean that components are registered as a side-effect?#2023-08-0112:10xificurCdom/div and friends mount a new DOM node as a side effect#2023-08-0112:48Dustin Getzthis is what I mean by effectful rendering in the London talk#2023-08-0112:49Dustin Getzthe effects are supervised by a functional effect system which provides strong guarantees wrt effectful evaluation that imperative languages do not#2023-08-0112:49Dustin Getz• cancellation
• concurrency and backpressure combinators
• composability / referential transparency – coordinate large scale effect fabrics
• glitch-free event propagation
• correct error handling by default (process supervision)
• resource lifecycle (RAII) – strong resource cleanup guarantees
#2023-08-0112:49Dustin GetzSee the talk for more discussion of this#2023-08-0113:00frozenlockI already watched your talk, but a bunch of stuff flew over my head because I lacked the knowledge/context to appreciate it.#2023-08-0113:00frozenlockI'll make sure to re-watch it once I'm more versed in the subject.#2023-08-0113:02Dustin GetzAh ok! TY for the feedback#2023-08-0112:25frozenlock(defonce conn ; state survives reload
This is on the server side. I would expect the ns to be evaluated only once, yet the comment mentions something about surviving reloads.#2023-08-0112:35xificurCelectric code needs to stay in sync on the server and client side, which means the whole ns is often reloaded#2023-08-0112:44Dustin Getzelectric uses a custom hot code reloading strategy: we hacked shadow compilation so that whenever a cljc namespace with electric code in it is recompiled, the server namespace is simultaneously reloaded #2023-08-0112:44Dustin Getzthis prevents spooky undefined behavior when a fresh client DAG connects to a stale server DAG#2023-08-0112:45frozenlockMakes sense, it's just unexpected when coming from... well, the rest of the Clojure world. 😉#2023-08-0112:47Dustin Getzyes :)#2023-08-0112:39telekidDustin, I would love to read a blog post similar to https://tonsky.me/blog/datascript-internals/. #2023-08-0113:12Dustin Getzhave you seen https://hyperfiddle.github.io/#/page/electric%20internal%20architecture#2023-08-0122:40telekidYeah, great doc#2023-08-0117:13joshchoOne feedback regarding the tutorial is that it will be nice if I can see the code and the writing at the same time. The writing references the elements of the code, but if I scroll down, I can't see the code. One possible (not great) solution is to have scrollable writing horizontally next to the code/result, but then the result window will be constrained horizontally. My viewport is a 1440p monitor.
Also, it would be really neat if hovering over the symbols highlighted that which is in the source code. Is the tutorial itself hosted somewhere? It'd be nice to play around with it.#2023-08-0119:10Dustin Getzdoes two windows solve your problem?#2023-08-0119:12Dustin GetzHere is the tutorial repo, is that what you're looking for? https://github.com/hyperfiddle/electric-examples-app#2023-08-0200:06joshchoYes, tyty.#2023-08-0117:31Garrett HopperDoes Electric's reactivity model in CLJS depend on JS object reference equality?
I have a function which gets run with an input JS array and returns the same array with some items added/removed.
It looks like none of the downstream dependencies of this variable get re-run. (I'm guessing because of work-skipping assuming the JS array hasn't changed since it's the same object?)
(It works as expected if I generate a new JS array instead, although that's not an option in my use case unfortunately.)#2023-08-0117:48Garrett HopperE.g.
(defn update-js-array [_ js-array]
(.push js-array #js {})
js-array)
(e/defn Example [input]
(let [js-array (array)
updated (update-js-array input js-array)]
(js/console.log (count updated) updated input)))
Whenever the input changes, the (count updated) will always log 1, whereas the actual updated array will be logged which shows its actual size has changed.#2023-08-0119:14Dustin GetzYes, the reactive evaluation uses clojure.core/= to decide if it can skip work#2023-08-0119:16Dustin Getza dumb hack could be to wrap the returned array in a unique wrapper object#2023-08-0119:16Garrett HopperJS wrapper object you mean? Otherwise it seems like it'd be the same issue#2023-08-0119:16Dustin Getzyes#2023-08-0119:17Garrett HopperGotcha; I hadn't considered that.
I only tried a CLJ wrapper. 🙂
Good enough for me without dropping down to some sort of missionary flow workaround 🤷#2023-08-0119:17Dustin Getzyou can also cachebust with a counter (assuming you control all reference writers)#2023-08-0119:19Garrett HopperThat's kind of how I'm doing it now, though having to keep the JS array and its associated cachebuster together and used together everywhere isn't quite as clean as just wrapping in a new JS object.#2023-08-0119:19Dustin Getz(defn ->Object "allocate a process-unique identity that cannot collide" []
#?(:clj (Object.) :cljs (js/Object.)))
#2023-08-0119:21Dustin Getz{(->Object) ref} i think would work, you just have to remember to pass around the composite and remember to unbox only from Clojure#2023-08-0120:59frozenlockWhat happens when you use defn instead of e/defn ? The function won't be reactive?#2023-08-0121:00Dustin GetzYes you get Clojure semantics not Electric. Same if you use clojure.core/fn inside an Electric function – you get clojure semantics inside the clojure function#2023-08-0121:00Dustin Getzelectric functions can call clojure functions; clojure functions cannot call electric functions#2023-08-0121:01Dustin Getzmagic network only works in electric functions#2023-08-0121:02frozenlockOh I see, so I can't use e/server inside a defn ?#2023-08-0121:03Dustin Getzcorrect#2023-08-0121:30frozenlockIs the new necessary here? https://github.com/hyperfiddle/electric-examples-app/blob/main/src/user/demo_todos_simple.cljc#L34
Could it have been (F. v) ?#2023-08-0122:46Garrett HopperYes, you can 🙂
(F. v) just desugars to (new F v); doesn't matter as long as F is bound to an Electric function#2023-08-0122:47frozenlockGreat, thanks#2023-08-0215:11frozenlockThat's a great video!
https://www.youtube.com/watch?v=tV-DoiGdUIo
I should have started with this instead of reading missionary's docstrings 😅#2023-08-0215:37telekidThis really tripped me up (and was hard to isolate.) From what I can tell, if an e/unmount thunk / expression refers to an electric binding, then the callback will be invoked whenever that binding changes. Is this expected / am I missing something? How can I ensure the callback is only run on unmount?#2023-08-0215:48Dustin GetzThanks, let me reproduce. It sounds like e/snapshot may be a workaround#2023-08-0215:48Dustin GetzI need to look more closely though#2023-08-0215:49telekidappreciate your help!#2023-08-0215:55xificurCwrapping the function (`(e/snapshot #(println ...))`) should do the trick#2023-08-0216:04telekidwrapping the thunk fails with Electric code (hyperfiddle.electric/snapshot) inside a Clojure function, but wrapping the dependency in e/snapshot and then passing that as an argument seems to work#2023-08-0216:09telekidThe fix, for reference#2023-08-0216:24Dustin Getzyou can snapshot the closure itself as well#2023-08-0216:25telekidIn case someone runs into this issue in the future. In my real code, I'm interfacing with an imperative javascript API. I pass an electric value to that API, and get a handle back that I can invoke later to clean up values created by the imperative API.
For me, the real fix wasn't actually to call e/snapshot on the returned handle. It was instead to call e/snapshot on the electric value that I pass to the imperative API. This keeps said API from being invoked every time the electric value changes, meaning I don't create multiple copies of that object; it just so happens to also prevent me from creating unnecessary disposal handles that flow into the on-unmount callback.#2023-08-0216:27telekid> you can snapshot the closure itself as well
Can't quite get this to compile. (I'm unblocked, just mostly curious now.)#2023-08-0216:47Dustin Getz@UABU2MMNW i dont think this is a bug, i think :id is the wrong keyfn#2023-08-0216:48Dustin Getzthat didn't fix it actually, i am still confused by the behavior, investigating further#2023-08-0216:49telekid> i think :id is the wrong keyfn
Ah sorry, I refactored the demo and missed that#2023-08-0216:49telekidbut yeah should reproduce with identity#2023-08-0217:17Dustin GetzOk I understand the data flow now, i think it's working as designed. But I understand that it is surprising. Can you tell me exactly what you expected to happen? Also perhaps can I get a screenshot of the actual code you have (to understand what you want to happen)#2023-08-0217:43telekidThis is the most relevant part of my real code. Note the call to e/snapshot, which fixed my issue. (`ap/create` creates a new asciinema-player object, source contains animation data, options contains config data for the player.)#2023-08-0217:47Dustin GetzHere is a demonstration of the intended usage, try this out#2023-08-0217:48Dustin GetzYeah I think your use of e/snapshot is ultimately correct, note you also need to snapshot the options#2023-08-0217:50Dustin GetzOne might ask, why not snapshot automatically? But that damages precision, sometimes you want to run cleanup on each change#2023-08-0217:50Dustin Getzperhaps "unmount" is not the best word for this concept#2023-08-0217:51telekidyeah I was thinking the same thing – this may be a naming (and doc) issue#2023-08-0217:51telekidunmount will invariably confuse poor old souls like me who have spent a lot of time in react#2023-08-0217:52telekid(now I'm realizing I probably don't quite know how e/on-unmount is supposed to behave)#2023-08-0217:52Dustin Getzin missionary this is called flow cancellation, i labeled it unmount to connect with React intuition but you're right to question it#2023-08-0217:53telekidfwiw where there is a conceptual link between electric and missionary, I think it's worth maintaining / highlighting it#2023-08-0217:54telekid(but that's obvi a matter of taste)#2023-08-0218:58Dustin GetzOk, I can now explain it better#2023-08-0218:58Dustin GetzUnfortunately I can only explain by first pointing at the macroexpansion
(defmacro on-unmount "Run clojure(script) thunk `f` during unmount. Standard electric
code runs on mount, therefore there is no `on-mount`."
[f] `(new (m/observe (cc/fn [!#] (!# nil) ~f)))) ; experimental
#2023-08-0218:59Dustin Getz#2023-08-0219:00Dustin Getzhere, I have now inlined the macroexpansion underneath the example call#2023-08-0219:01Dustin GetzThe thunk returned by m/observe is the cancellation callback, afaik this is exactly the same as React.useEffect#2023-08-0219:01Dustin GetzWhat's happening is the cancellation callback is rebuilt when v or visible changes, which cancels the m/observe, calling the former callback.#2023-08-0219:01Dustin GetzThis is surprising because one would reasonably expect changing v or visible to be an "update" not a "unmount"#2023-08-0219:02Dustin GetzHaving understood the internals one can now use e/snapshot appropriately to defend this. I will need to think about a better way to express what is intended here that doesn't require internals knowledge to use#2023-08-0313:27telekidcool, this makes sense. Thanks Dustin!#2023-08-0216:45joshchoHas anyone gotten something like https://github.com/nextjournal/clerk to behave nicely with electric e/defn?#2023-08-0217:43Dustin GetzI explored this but was unhappy with the result#2023-08-0217:44Dustin Getzwhat do you want to do exactly? I have a secret project that might be related#2023-08-0217:44Dustin Getzpowering https://dustingetz.electricfiddle.net/electric-fiddle.essay!Essay/electric-y-combinator#2023-08-0218:01joshchoI might go clerk-less, but following is a rough specification. It's simply a realization of https://youtu.be/PUv66718DII?t=394.
Given some Electric component X, I want to see side-by-side:
1. Rendering of X
2. Non-editable, interactive source code of X
The interactive source code of X would have the following affordances:
1. Hovering over the opening or closing parenthesis of some S-expression Y means Y is "focused" in the rendering of X. Every element in X but not in Y would fade out. This would be much like browser inspectors.
2. Clicking a let-bound variable would make the variable into an atom, allowing control over that variable. Then given the assumed data type of the variable, display a slider/input that facilitates fine-grained control over that variable.
I am not sure exactly how I will do this, so exploring atm.#2023-08-0306:26teodorluI'm also curious about the intersection of Clerk and Electric.
I feel that Clerk nails the interactive experience while developing, while electric nails the model for writing interactive applications.#2023-08-0308:13joshchoI am thinking... use Electric to build elaborate viewers for Clerk?#2023-08-0308:14joshchoThen we can use these viewers to develop Electric programs... profit.#2023-08-0312:56teodorluYes! That’s my intuition too.
Hyperfiddle does currently require more than Clerk; you need a shadow-cljs cross backend+frontend project. But I think that weight would probably be worth it for me. I want to gain very good perception into the problems I’m working on. I want to use Clerk to tell linear stories and visualize graphs, and I want to use Electric to create backoffice apps.
Though — I’m not sure if it makes sense for either project (Electric or Clerk) to take on the full weight of the other project. So perhaps the best thing to ask for is that each project is aware of the nice parts of the other project.
But that’s my intuition, and I didn’t have a hand in making either project.#2023-08-0313:07joshchoI am thinking two things:
1. Write Clerk viewers in Electric (but maybe convert it later to reagent or sth for portability after exploration phase).
2. Make it easy to see Electric e/defn'd symbols in Clerk (which takes some effort rn, but probably possible).
I think whatever comes of this, it should be built on top of Electric and Clerk, and not be part of either one. As you said, I just want to gain a really good perception of the problems I am working on via Clerk. I want better sight (which Electric provides as well). I also want ergonomic/haptic controls over the programs (like why am I typing "7", "8", 9", when I could have a slider?). Better affordances.
I am tied heavily to Emacs though, so maybe have these affordances in the Clerk view? That's my current thought process anyway.#2023-08-0313:17teodorluHave you seen how @U017QJZ9M7W uses Clerk to write viewers in #C01ECA9AA74? Plenty of sliders for controlling physics simulations in there!#2023-08-0313:58joshchoExtremely cool. I will take a look.
It's very odd to me that these ideas have not taken hold in the wild in, e.g. VS Code. It's not that surprising to use interactive controls for variables, and also Bret Victor was demonstrating it in his talk. People aren't interested in tools? At least that's my explanation.#2023-08-0314:01Dustin Getzmy hot take is none of Bret Victor's ideas can be implemented in production web applications without first having Electric#2023-08-0314:26joshcho+1 to that hot take. what we needed was a medium for medium's (and the medium for the medium of medium's being Clojure...)#2023-08-0314:28joshcho@U09K620SG At least theoretically, how much of Electric's ideas owe being hosted on Clojure? Do you think Electric's ideas could have matured to this degree with another language?#2023-08-0315:45Dustin GetzUnclear how to implement Electric outside of Clojure. We could see like a 60% good solution being done with Typescript#2023-08-0217:09joshchoAlso, this is a bit of a long shot, but is it possible to get the source code of a function symbol defined via e/defn?#2023-08-0217:40Dustin Getz#2023-08-0217:40Dustin Getzi think i saw a better way somewhere, but this does work#2023-08-0217:42Dustin Getzon electric vars too, which are just clojure vars#2023-08-0218:01joshchoI might go clerk-less, but following is a rough specification. It's simply a realization of https://youtu.be/PUv66718DII?t=394.
Given some Electric component X, I want to see side-by-side:
1. Rendering of X
2. Non-editable, interactive source code of X
The interactive source code of X would have the following affordances:
1. Hovering over the opening or closing parenthesis of some S-expression Y means Y is "focused" in the rendering of X. Every element in X but not in Y would fade out. This would be much like browser inspectors.
2. Clicking a let-bound variable would make the variable into an atom, allowing control over that variable. Then given the assumed data type of the variable, display a slider/input that facilitates fine-grained control over that variable.
I am not sure exactly how I will do this, so exploring atm.#2023-08-0309:44braai engineerWhat's the easiest way to deploy an Electric Clojure app + Datomic Transactor to AWS instance with HTTPS like http://Fly.io but without Fly? (too unreliable) ECS Fargate + Terraform? (Expensive with all the ALBs and blablah.)
Not trying to do full-on production ready - just a test box that has all the things.#2023-08-0309:45braai engineerUnrelated to Electric but I want this because I deployed an EC2 box running my Uberjar + Datomic Transactor but the thing keeps keeling over at exactly midnight in my timezone (sometimes other times too). Thought it was memory issue during some vacuum process so I uninstalled dokku but still happens.
I see this error in journalctl but don't have time to figure out why:
Aug 01 19:16:55 ip-172-31-16-20 java[6231]: WARN datomic.coordination: ... caused by ...
Aug 01 19:16:55 ip-172-31-16-20 java[6231]: com.amazonaws.SdkClientException: Unable to execute HTTP request: dynamodb.af-south-1.a>
Aug 01 19:16:55 ip-172-31-16-20 java[6231]: at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException>
Aug 01 19:16:55 ip-172-31-16-20 java[6231]: at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttp>
Aug 01 19:16:55 ip-172-31-16-20 java[6231]: at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClie ...
EC2 Status checks start failing at the same time and I can't SSH into the box - have to reboot it from EC2 console. Then all is well again for a few days.#2023-08-0309:56henrikWhile it’s not necessarily nice in the long run for a number of reasons, EBS is fairly easy (by AWS standards) to get started with.#2023-08-0309:57braai engineerEBS = block storage? Do you mean ECS?#2023-08-0309:57henrikSorry, Elastic Beanstalk#2023-08-0309:57braai engineerWhat would be the long-term solution? My own EC2 instances behind a load balancer?#2023-08-0310:02henrikThere are a variety of options. Kubernetes on EKS, to get the biggest hairball you’ve ever seen, and nightmares for the rest of your life, but you want to be industry compliant.
Custom Terraform + Nomad (Traefik as load balancer), if you want something with far less complexity and ease of local development.
There are other options too, but I can’t vouch for them because the above is what I have experience with.#2023-08-0310:23Dustin Getzseriously rent a linux box and ssh in#2023-08-0310:24Dustin Getzif you don’t have 10,000 customers this cloud stuff is a waste of time#2023-08-0310:41henrikThough Beanstalk isn’t that hard, and you get something that you can throw a JAR at.#2023-08-0311:56Dustin Getzwould beanstalk host transactor?#2023-08-0314:07braai engineerfor now I have just started a t3.medium on EC2 (because DynamoDb) with Dokku on it#2023-08-0315:23Brandon StubbsFor a cheap compute instance:
https://www.linode.com/estimator/
https://www.digitalocean.com/pricing/droplets
Or take advantage of one of the main cloud providers free tiers (Also AWS you can use a spot instance, doesn’t matter if you go down for a couple minutes if it’s dev work)#2023-08-0315:24Dustin Getz@U051SPP9Z what do you need dynamo for? Datomic query performance is the same regardless of disk storage or dynamo due to the immutable caches, dynamo is for scaling the transactor#2023-08-0315:31braai engineerSurely the peers don't have the entire set of indices in-memory at all times?#2023-08-0315:32Dustin Getzat what scale? most datomic dbs i've seen at Series A/B fit into ram#2023-08-0315:32Dustin Getzand SSD is like 1ms away, dynamo is farther#2023-08-0315:32markaddlemanIIRC, Datomic maintains a cache on-disk using EFS. That cache is available to all instances within a query group. There is some latency reading the EFS cache but it’s not too bad. Hot pages are kept in memory#2023-08-0315:33braai engineerat my scale probably all fits in RAM. Should I use H2 instead? I don't want to pay for RDS.#2023-08-0315:33braai engineerI was under impression that Peer would stream data from Dynamo as needed and cache heavily, but not that all datoms would be in memory all the time.#2023-08-0315:35braai engineerI used Dynamo coz it's cheap and scales#2023-08-0315:36markaddlemanI haven’t used Datomic and H2 but I have used H2 under heavy load in the past and I was impressed with it. You’ll probably get some good advice from folks in #C03RZMDSH#2023-08-0315:36Dustin Getzjust use local hard disk#2023-08-0315:37braai engineercom.datomic/dev-local?#2023-08-0315:37Dustin Getzuse datomic pro not datomic cloud#2023-08-0315:37braai engineeryeah I'm on pro#2023-08-0315:37braai engineerbut only one transactor with dev-local, no?#2023-08-0315:38Dustin Getzdev-local is datomic cloud#2023-08-0315:38Dustin Getzyou don't need a high availability configuration#2023-08-0315:38Dustin Getzyou need: linux box, hard drive, cron job for nightly backup#2023-08-0315:38Dustin Getzthat's it#2023-08-0315:40braai engineerwhat's the downside to Dynamo if I already have it set up from the CloudFormation templates?#2023-08-0315:45markaddlemanFWIW, I used Datomic Cloud and found it very nice to work with. I recall having a few issues setting it up and getting it to scale but, again, this was several years ago. Those issues may have been worked out. Once set up, it was a breeze#2023-08-0315:45braai engineerElectric does not work with Datomic Cloud#2023-08-0315:45braai engineerI have Pro Transactor on DynamoDb setup via CloudFormation templates#2023-08-0315:46markaddlemanOh, my bad.#2023-08-0315:46markaddlemanI can’t speak to Pro#2023-08-0405:33nakkayaNoticed this last night, I had the following,
(dom/on "click" (e/fn [_]
(e/server
(db/create-collection
(h/user.)
(e/client (update-vals coll deref))))
(h/reset-form "newCollectionForm")
(swap! !dirty-collection inc)))
This worked as expected, db call returns form resets, atom is swapped and UI is updated. Refactoring the above into,
(dom/on "click" (e/fn [_]
(let [u (h/user.)
c (e/client (update-vals coll deref))]
(e/server
(db/create-collection u c)))
(h/reset-form "newCollectionForm")
(swap! !dirty-collection inc)))
did not work, what happened is atom is swapped before the db call returns UI does get update but can not pickup the new record since it is not inserted into the db yet.#2023-08-0407:25xificurCThe body of the e/fn is an implicit do, which runs all branches concurrently. This means the swap! will almost certainly finish before the server call finishes. If you want to swap! only after the db call finishes you can sequence them with case
(case (db/create-collection ...)
(swap! !dirty-collection inc))
Another thing I noticed - in your first version h/user is called on the server, whereas in the refactored one it's on the client. Is that intended?#2023-08-0408:04nakkayaSo in this case my first form is working by accident? Since both forms has 3 top branches. (h/user.) runs (e/client ...).#2023-08-0408:05xificurCyes, most likely. If you want the 2 other branches to run after the db update you should (case a (do b c))#2023-08-0408:08nakkayaIs there a particular reason for the case why not a when or wrapping everything in a do ?#2023-08-0408:09xificurCit's just the simplest form that works and doesn't care about the return value of a, it can be false or nil. With when you'll need to make sure the condition holds with e.g. (when (do (e/server (create-collection)) true) ...)#2023-08-0408:10xificurCwrapping everything in a do is what you have now, function bodies are implicitly wrapped in a do#2023-08-0408:12xificurC(do a b) mounts a and b in order and any async work they do will run concurrently. (case a b) mounts a and will only mount b once it resolves to a non-exceptional value. Does that make sense?#2023-08-0408:16nakkayayeap thanks for the clarification.#2023-08-0407:49teodorluHypothetically — if one wanted to make a markdown editor with Hyperfiddle and pandoc;
• edit raw markdown on the left (textarea), preview pandoc generated HTML on the right,
• edits are synced back to the file system,
• a left sidebar allows the user to select which markdown file to edit,
how would one “debounce” the save events? I don’t want each on-change to trigger a new file save, I’d like the app to wait until a second or so has passed to save. Any recommendations on an Electric-idiomatic solution to this?#2023-08-0408:01xificurCthe off-the-shelf solution would be to https://github.com/leonoel/missionary/wiki/Debounce-and-Throttle#2023-08-0408:02xificurCyou can turn a reactive value x into a missionary flow with (e/fn [] x) and the missionary flow back to electric with new
(->> (e/fn [] x) (debounce 1000) new)
#2023-08-0408:31teodorluHuh, that simple. Thanks! 💯#2023-08-0410:15Dustin Getzwhy do you want to delay the save?#2023-08-0410:15Dustin Getzwhy not stream the result to the server as fast as possible which is strictly more reliable#2023-08-0411:26henrikOTOH, you might not necessarily want to hammer the DB, or put a potentially huge value on the socket per character stroke with only minor incremental differences in each payload.
But if you were using Yjs or Automerge, you could stream the emitted diff events, aggregate them in memory, server side (or something distributed), and commit a compressed snapshot every so often to the DB.#2023-08-0411:35henrikThis is something I’ve planned to look into btw. I imagine Electric will be very convenient when dealing with CRDT libs.#2023-08-0416:17teodorlu> why do you want to delay the save?
> why not stream the result to the server as fast as possible which is strictly more reliable
I assumed spit-ing the whole file on each keystroke would be a bit much? Perhaps not? I haven’t measured whether this is actually necessary, so perhaps I’m just underestimating disk speeds.
(defn random-file [n]
(apply str (repeatedly n #(rand-nth (vector "abcdef\n")))))
;; assume 10 tweets
(time (spit "/tmp/somefile" (random-file (* 10 300))))
;; "Elapsed time: 7.355833 msecs"
And I don’t type each 7 ms.
Good point. Thanks!#2023-08-0416:19teodorlu> But if you were using Yjs or Automerge, you could stream the emitted diff events, aggregate them in memory, server side (or something distributed), and commit a compressed snapshot every so often to the DB.
>
> This is something I’ve planned to look into btw. I imagine Electric will be very convenient when dealing with CRDT libs.
Agreed, electric is in a great position to leverage CRDTs.#2023-08-0416:20Dustin Getzspecifically you can delay the second effect but not the first effect, which is something different than debounce because it runs the effect on the leading edge#2023-08-0416:21teodorlu> delay the second effect
Does “delay the second effect” have a name?#2023-08-0416:21Dustin Getzthis also moves any latency damaging operation to be co located with the actual platform effect that can’t keep up, rather than debouncing the client for an abstract effect on the server#2023-08-0416:25teodorlu> Does “delay the second effect” have a name?
I mean, if “debounce” is “delay on the client”, does FRP / effect systems have a term for “i want to subscribe to effects, and run a save not more often than some time in milliseconds”?
As you said, it’s nice to get the data to the server as soon as possible.#2023-08-0416:26Dustin Getzi need to think about it#2023-08-0416:26Dustin Getzwe saw this structure before while developing electric but i have forgotten. iirc we left it as an open question whether debounce is ever appropriate#2023-08-0416:32teodorluLooks like the term “debouncing” is used in hardware for physical buttons. On the way down, the button “bounces”, and the raw signal will send several “on”/“off”/“on”/“off” cycles. Arduino has a https://www.arduino.cc/en/Tutorial/BuiltInExamples/Debounce.#2023-08-0512:36henrikIf you put n values on the wire only to throw away 90% of them server-side, it seems more frugal to throw them away as early as possible.#2023-08-0512:39Dustin Getzyou can diff immediately client side#2023-08-0512:40Dustin Getzfwiw electric v3 is differential dataflow, not clear exactly what the API will be if any but the language will offer advanced support for this to get differential network traffic in all cases not just e/for#2023-08-0513:06Dustin Getz> it seems more frugal to throw them away as early as possible
having thought about this a bit more, and i am not trying to be pedantic, i think this is a premature optimization even without the diffing. Bandwidth is free on desktop; on mobile the conserved resource is battery and the radio had to turn on with or without this optimization, do we have any data showing that battery will be worsened by streaming too much data if the radio is already on?
The real cost here is server scalability (can we reach 100k concurrent connections per server or merely 10k). And we don't have any data about that (likely best to let the Electric runtime own optimizations here until we know more, Electric itself is likely the dominant factor in server scalability today).
The electric wire protocol already optimistically streams tons of stuff before it is actually needed (program control flow mostly). The key is that it is not a request/response round trip; when we broadcast something we are simply allowing observers to be informed and therefore there is no latency impact to the user for broadcasting too much. Debouncing, however, is infuriating – Roam Research has this problem, typing into the command bar is lagged and the UX is damaged (it's infuriating)#2023-08-0513:06Dustin GetzCaveat, this is all "theorycrafted" (Starcraft video game reference) so probably wrong#2023-08-0513:10Dustin GetzAlso as @U3X7174KS noted, getting user inputs safely to the server asap is valuable because the server environment is much more stable than the user environment. Even if we delay the actual transact to avoid hammering the database — due to relational database limitations, not all dbs have a problem with being hammered — the edit is still safer on the server and can even be passed through a message queue#2023-08-0513:11Dustin Getz@U06B8J0AJ please let me know your reaction to this, this topic has been on my mind wrt how we design electric-ui5#2023-08-0513:14teodorlu(I really appreciate everyone’s takes on this—Peter, Dustin and Henrik 💯. It still feels alien to me that we can treat network sync as something cheap / free)#2023-08-0513:40henrikI understand the Electric frame of mind reaching towards “the network doesn’t exist”. For a toy program or small documents, or the fact that the focus of Electric currently is correctness—it’s probably fine. I’m not going to sign off on “therefore it’s going to be fine forever”, however.
If we’re talking about an intermediate state that will be replaced later on; yeah, no worries.
Let’s take one of the docs from my DB, which is 26kb (Prosemirror representation, as it happens). Let’s say 275 characters typed per minute (average for a non-expert adult, apparently) , and let’s say 100 active users. 26*275*100 = ~715mb/min, only from editing text. Still not the end of the world, but I would probably switch to a CRDT and sending diffs at that point, because it doesn’t look promising for scaling further.#2023-08-0513:46henrikI have to confess that this is also from an emotional PoV, just because I dislike the feeling of wasting resources if I can avoid it.#2023-08-0513:50henrikHaving said all that, my bet is that Electric will be better at managing resources that I am overall, and therefore will win over a lovingly handcrafted, free-range, artisanal network stack in the end.
The above just strikes me as something where I might have to help Electric along a bit, when at scale.#2023-08-0513:59teodorluYeah—for example by breaking documents into parts that can be reasoned about in isolation. If we can “sync a paragraph with a UUID”, it’s easier to gain good performance than if we need to “sync the whole document”.
If we want to sync a giant 3D scene from the browser to the server, I suspect we’ll want to transfer object IDs, object coordinates and camera position and rotation, rather than pixels.#2023-08-0514:04henrikYes, but also, I would probably recommend against breaking it into parts, because at that point you’re on your way to reinventing text CRDTs or operational transformation. I think the better path would be to pick something off the shelf, because those things can get complex quickly.
Also, the premature optimization point stands, I think. Diffing implies reconciliation, which can get tricky. Libs will solve the algorithm itself, but you still have to manage some form of code or infrastructure for it. This will be a lot of overhead for single-user applications, at least.#2023-08-0514:05Dustin Getzpresumably your data structure is smarter than just string buffer ?#2023-08-0514:10henrikMeaning what?
Prosemirror, for example, has individual structures for ranges of characters. You might have a paragraph, which might contain multiple structures, if you’ve used bold, italic or something else in it.
Let’s say you make each adressable with a UUID (which it isn’t by default, in Prosemirror structures), you can target an update to one of them. What do you do when the user removes bold, etc.? Now you’ve got to merge at least two structures into one.
It all just leads to headaches that are better left for someone (-thing) else to solve. I.e., send blobs or use Yjs/Automerge/etc.#2023-08-0514:11Dustin GetzJavascript "CRDT" libs are more than conflict-free data types with convergent merge operation, they also generally include network stuff, which should be treated orthogonally. Is the essence of CRDT really just a reactive data structure + a convergent merge operator?#2023-08-0514:14henrikAgree, it’s a completely separate concern, and should be treated as such. That’s how I would like to approach it with Electric.
But a lot of people want to sell you a whole stack, and preferably manage it for you, but you don’t have to buy into that.
The Automerge people wrote that they were planning on integrating the Peritext algorithm (https://www.inkandswitch.com/peritext/).#2023-08-0514:15henrikOh, and yeah, CRDTs know nothing of any network, in and of themselves.#2023-08-0514:17Dustin GetzRhetorical question: How does one implement a reactive collection? I point to McSherry /Materialized : the whole point of differential dataflow as presented by McSherry is to implement an efficient reactive relational JOIN between two very large collections/tables. I.e., differential dataflow is the answer to the opening question.#2023-08-0514:17Dustin GetzTherefore I think Electric v3 (Differential Electric) provides the exact primitives needed to implement such reactive collections. The only thing missing from "CRDT" is now just a convergent merge operator which you use instead of database transact – you get to decide if you want conflict-free or conflict-full writes.#2023-08-0514:19Dustin GetzI dont think it's hard either, i think we're talking like a few dozen lines to get an auto-diffed text editor data type that is perfectly network transparent#2023-08-0514:19henrikAnd the semantics for reconciling edits happening out of order is there?#2023-08-0514:19Dustin Getzthose semantics would be encoded into the convergent merge operator#2023-08-0514:22teodorlu> Yes, but also, I would probably recommend against breaking it into parts, because at that point you’re on your way to reinventing text CRDTs or operational transformation
Good point.#2023-08-0514:24henrikI see your point @U09K620SG. The text editing semantics are hard to get right though (see the Peritext article). Building the merge operator is not a small task.#2023-08-0514:24henrikThough of course, having Peritext native would be wild.#2023-08-0514:24Dustin Getzdo they provide an impl? I thought this was a solved problem#2023-08-0514:27henrikI’m not sure if there’s a reference implementation, or just the outline of the algorithm.
Depends on what you mean by solved. There are obviously implementations out there, and they all have different edge cases. Multiplayer and the time axis (an edit can arrive right now from a device that was offline for a month, which effectively changes the timeline of everything happening after). There’s still R&D happening.#2023-08-0514:41henrikThe other aspect is the efficiency of the algorithm and size of the payloads. Smaller payloads and more efficient algorithm translate to lower latency. This in turn translates into better UX in terms of the interactiveness/“liveness” of the application in a multiplayer setting. I.e., it’s not exactly about resource efficiency as much as it is about the impact on the experience.
(Btw., not talking network here, just how quickly we can “roundtrip” an edit)
This is also an area of ongoing R&D, and there are optimizations specific to text CRDTs. My interpretation is that this is why CRDT libs have a bit of a benchmark war, and why anyone cares about those benchmarks.#2023-08-0514:54Dustin Getzthat applies to ordinary reactive programming as well, it is a central problem to electric itself#2023-08-0514:55Dustin Getzwell, hmm, i see a vector to challenge that, i'll leave the assertion though#2023-08-0515:06henrikYes, this is just to say that multiplayer text is a highly interactive, rapidly changing part of an app, and as such, performance improvements that would be absolutely marginal in the rest of the app can make a difference here. Which is why people have put efforts into it. None of that implies that it’s not good to look for more general, less text-centric ways to provide that. I think that’s what most are aiming for.#2023-08-0515:37henrikRe: an earlier question
@U09K620SG https://automerge.org/docs/hello/#design-principles
> Network-agnostic. Automerge is a pure data structure library that does not care about what kind of network you use.
#2023-08-0418:29Vincenthttps://news.ycombinator.com/item?id=36957204 is this like hyperfiddle but not?#2023-08-0512:03henrikNo, it’s based off Prosemirror. Collaboration is provided via Yjs. It’s not a general, agnostic framework like Electric.
We use Tiptap at work, and I’m not a fan. It has weird, shared state where manipulating one editor might change things in another one, if on screen simultaneously.#2023-08-0422:13joshchoI have a button with a value (like 10) and when the button is clicked, a range appears. I want the range to disappear when anywhere outside the button is clicked. What's the best way to manage this?#2023-08-0422:15joshchoThis is the current code. It's nice and local, but I imagine if I want a global listener with state management, then I would need some way for the global listener to access !showing atom. This is not a problem, but I have several of these buttons, leading to more complications. It would be best if the logic for un-selecting can be done within this code block, like (dom/on-outside "click" (e/fn [v] (reset! !x v))) which would register a global listener for any clicks outside of this element.
(let [!showing (atom false)
showing (e/watch !showing)
!x (atom 10)
x (e/watch !x)]
(let [value (int value-string)]
(dom/span
(ui/button
(e/fn []
(swap! !showing not))
(dom/on "click" (e/fn [e] (.stopPropagation e)))
(dom/b
(dom/text x)))
(ui/range
x
(e/fn [v]
(reset! !x v))
#_(dom/on-outside "click" (e/fn [v] (reset! !x v)))
(dom/props {:min (- value 50)
:max (+ value 50)
:value @!x
:class (str "absolute "
(when-not showing
"hidden"))})))))#2023-08-0422:39braai engineerBeen down this road: put a click handler on the parent/container div. Keep track of any open pickers in state (!open-pickers), and when container is clicked, close open pickers. You will need to catch clicks on divs above with (.preventDefault e) to prevent it bubbling "down" to parent if you want to keep them open.#2023-08-0423:15joshchoWas reading the dom/on source. I think it may be possible to dynamic-let a container div, and set the dom/on in the child.#2023-08-0504:50joshchoIt's actually possible to localize the "click outside for unselect" logic, with just one e/def for container-node. You don't need to mess with a whole sleuth of open pickers.
(e/def container-node)
(dom/div
(let [container-node dom/node]
(e/for [value [1 2 3 4 5]]
(let [!showing (atom false), showing (e/watch !showing)
!x (atom value), x (e/watch !x)]
(let [value (int value-string)]
(dom/span
(ui/button
(e/fn []
(swap! !showing not))
(dom/on "click" (e/fn [e] (.stopPropagation e)))
(dom/text x))
(ui/range
x
(e/fn [v]
(reset! !x v))
(dom/on container-node "click"
(e/fn [e]
(reset! !showing false)))
(dom/on "click" (e/fn [e]
(.stopPropagation e)))
(dom/props {:min 0
:max 10
:value @!x
:class (str "absolute "
(when-not showing
"hidden"))}))))))))
Pretty cool! You can achieve a similar thing in React, but the Electric version is much more explicit.#2023-08-0506:04joshchoI made a prototype of something I discussed https://clojurians.slack.com/archives/C7Q9GSHFV/p1690999313930169?thread_ts=1690994711.817419&cid=C7Q9GSHFV: I wanted interactive controls over any Electric code, which would reactively change the display. Pretty new to Electric, would appreciate any feedback; the gist file is https://gist.github.com/joshcho/364bdd8cfd6d33e2f5f1782d90e944aa. The low LOC for expressing ideas is pretty mind-boggling.#2023-08-0510:29Dustin Getzamazing#2023-08-0513:07teodorluWow!#2023-08-0517:07phronmophobicJust saw that @U066L8B18 cross-posted this in #visual-tools. Cool stuff!
You may also be interested in https://dl.acm.org/doi/10.1145/3428290 which explores a generalized version of this idea.#2023-08-0509:15braai engineerI struggle to parse these errors in RCF:
❌ FAIL in () (form-init10980895378543017097.clj:223)
can fetch token
expected: (:= (fetch-session (d/db c) token) {:db/id (quote _), :session/token token, :session/ticket (quote _), :session/created-on (quote _)})
actual: "Failed to unify :session/token and :db/id in [:session/token]"
What's missing? What's there that shouldn't be there? I have to manually prn out the thing I'm testing and then diff it in my head.
How can I get more readable output?#2023-08-0510:25Dustin Getzpaste the test please#2023-08-0510:33braai engineer(tests
(du/with-fresh-connection c
"seed db"
@(d/transact c schema/v1-seed)
(defn touch-entity [db eid]
(some-> (d/entity db eid) (d/touch)))
;; Pull should do the same, but annoying nil check:
(defn pull-entity [db eid]
(let [ent (d/pull db '[*] eid)]
(if (= {:db/id nil} ent)
nil
ent)))
(defn create-session! [conn]
(let [token "abc" ;; usually random
now (java.util.Date.)
{:as rx :keys [db-after tempids]}
@(d/transact c [{:db/id "ticket"
:ticket/timestamp now}
{:db/id "session"
:session/token token
:session/ticket "ticket"
:session/created-on now}])]
{:session/id (d/resolve-tempid db-after tempids "session")
:session/token token
:ticket/id (d/resolve-tempid db-after tempids "ticket")}))
(let [{:as session
session-id :session/id
token :session/token
ticket-id :ticket/id} (create-session! c)
db (d/db c)]
"pull works"
(pull-entity db [:session/token token]) := {:db/id session-id
:session/token token
:session/ticket {:db/id ticket-id}
:session/created-on _}
"touch fails. why?"
(touch-entity db [:session/token token]) := {:db/id session-id
:session/token token
:session/ticket {:db/id ticket-id}
:session/created-on _})))
=>
seed db pull works touch fails. why?
expected: (:= (touch-entity db [:session/token token]) {:db/id session-id, :session/token token, :session/ticket #:db{:id ticket-id}, :session/created-on (quote _)})
actual: "Failed to unify :session/token and :db/id in [:session/token]"
=> false#2023-08-0510:50braai engineerI don't understand RCF. What does _ placeholder mean? Why can't it unify?
{:db/id 17592186045435, :session/token "abc", :session/created-on #inst "2023-08-05T10:50:23.013-00:00", :session/ticket #:db{:id 17592186045434}, :session/owner :petrus}
❌ FAIL in () (datomic.clj:247)
missing session returns nil can create and fetch a session. session has ticket. can fetch session by token claiming missing session throws claiming a valid session:
1. upserts WA contact,
2. updates ticket (with what?)
3. notifies owner
expected: (:= claimed-session #:session{:token (quote _), :ticket (quote _), :created-on (quote _)})
actual: "Failed to unify :session/created-on and :session/ticket in [:session/created-on]"#2023-08-0510:52braai engineerWhat am I misunderstanding about RCF := ?
(let [session (fetch-session (d/db c) token)
claimed-session (claim-session! c token :petrus)] ;; claim returns session with :session/owner assoc'd.
claimed-session := (assoc session :session/owner :petrus))
=> expected: (= claimed-session (assoc session :session/owner :petrus))
actual: (not (= {:db/id 17592186045435, :session/token "abc", :session/created-on #inst "2023-08-05T10:52:22.409-00:00", :session/ticket #:db{:id 17592186045434}, :session/owner :petrus} {:db/id 17592186045435, :session/token "abc", :session/created-on #inst "2023-08-05T10:52:22.409-00:00", :session/ticket #:db{:id 17592186045434}, :session/owner :petrus}))#2023-08-0510:56braai engineeralso fails if I use def:
(tests
...
(def session (fetch-session (d/db c) token))
(def claimed-session (claim-session! c token :petrus))
; claim-session! returns the session above but with :session/owner assoc'd.
claimed-session := (assoc session :session/owner :petrus))
=> same error as above.
#2023-08-0512:15Dustin Getzthis is all very disconnected and hard to follow#2023-08-0512:16braai engineerkey issue seems to be (d/touch (d/entity db eid)) returns a map that fails equality in RCF, but prn output seems to be the same#2023-08-0512:17braai engineerworks if I use d/pull#2023-08-0512:18braai engineerit is an auth flow:
1. an anonymous session is created when a user does action on website
2. user claims session via an auth mechanism (WhatsApp)
3. session should now have :session/owner = user-ref
When I try to test for it, it fails. But it seems to fail because (d/touch (d/entity ..)) has different equality, or I don't understand how to use RCF correctly.#2023-08-0512:20braai engineerthe crux is this message:
expected: (= claimed-session (assoc session :session/owner :petrus))
actual:
(not (= {:db/id 17592186045435,
:session/token "abc",
:session/created-on #inst "2023-08-05T10:52:22.409-00:00",
:session/ticket #:db{:id 17592186045434},
:session/owner :petrus}
{:db/id 17592186045435,
:session/token "abc",
:session/created-on #inst "2023-08-05T10:52:22.409-00:00",
:session/ticket #:db{:id 17592186045434},
:session/owner :petrus}))#2023-08-0512:21braai engineernote that the maps are equal#2023-08-0512:21braai engineermy best guess is that :session/ticket has different type when returned via d/touch than via d/pull#2023-08-0512:24Dustin Getzissues with d/entity equality are likely a datomic bug#2023-08-0512:24braai engineerright but I do (d/touch) on it#2023-08-0512:25Dustin Getzuse (into {}) to get rid of datomic types#2023-08-0512:28braai engineerhmm same issue. this works:
(d/pull db-after '[*] session-id) ;; or (into {} (d/pull db-after '[*] session-id))
but this fails the test:
(into {} (d/touch (d/entity db-after session-id)))
so weird. when I prn the result - exactly the same.#2023-08-0512:29braai engineerhmmm#2023-08-0512:30braai engineerok now I'm seeing that d/touch resolves :db/ident. I thought I checked this#2023-08-0512:32braai engineerOK, so _ in RCF only works in map literal?#2023-08-0512:35braai engineer(let [session (fetch-session (d/db c) token)
claimed-session (claim-session! c token :petrus)
expected (assoc session :session/owner :petrus) ;; claimed session has new owner
actual claimed-session]
(prn "expected: " expected)
(prn "actual: " actual)
claimed-session := expected)
now let's look at the prn output:
"expected: " {:db/id 17592186045435, :session/token "🇲🇴☝🏿💆🏻🚼🚊", :session/created-on #inst "2023-08-05T12:32:51.711-00:00", :session/ticket #:db{:id 17592186045434}, :session/owner :petrus}
"actual: " {:db/id 17592186045435, :session/token "🇲🇴☝🏿💆🏻🚼🚊", :session/created-on #inst "2023-08-05T12:32:51.711-00:00", :session/ticket #:db{:id 17592186045434}, :session/owner :petrus}
expected and actual are identical (using touch).
so why is := equality check failing? If I get rid of the :session/owner assoc it passes, so it must be ref entity magic related.#2023-08-0512:37braai engineerk, if I switch to checking the value of :db/id directly then it works.#2023-08-0512:37Dustin Getzi suspect d/enty equality, please use (into {})#2023-08-0512:37Dustin Getz"prints exactly the same" is not the same as "equals"#2023-08-0512:37braai engineeryeh I tried that (into {} (d/touch (d/entity ..)) does not help.#2023-08-0512:37braai engineeryeah#2023-08-0512:38braai engineerinto {} does not change the entity magic#2023-08-0512:38Dustin Getzi've seen exactly this before in our own tests#2023-08-0512:38braai engineerI'm just avoiding touch now#2023-08-0512:38Dustin Getzif you sus[ect RCF bug at this point please isolate a minimal repro#2023-08-0512:39braai engineerI was trying to do this:
{:session/token "abc"
:session/owner {:db/id 123456}} := (assoc {:session/token "abc"} :session/owner _)#2023-08-0512:39braai engineerbut _ will only work in map literal#2023-08-0512:42Dustin Getzyou can't assoc a hole into a map#2023-08-0512:43Dustin Getzthat's not how clojure works#2023-08-0512:44braai engineeryeah 😕#2023-08-0517:15Dustin GetzFound my repl transcript of this issue - https://github.com/hyperfiddle/electric/blob/a4e806286e549ecdb1d672194547aca3852fcbf0/src-dev/dev.cljc#L112-L137#2023-08-0517:15Dustin Getzyou're right, (into {}) is insufficient, i misremembered sorry#2023-08-0517:16Dustin Getzi think because :db/id is omitted by (into {})#2023-08-0517:16Dustin Getzalso nested refs#2023-08-0517:16braai engineerwhuuu. plz can we get Rich on this#2023-08-0517:16braai engineerthx for checking#2023-08-0517:17Dustin Getzim not sure about the :db/id thing, its unclear from reading why (into {}) doesn't work#2023-08-0517:17Dustin Getzif you figure out why i will update this and move it someplace better#2023-08-0517:17Dustin Getz(I am working on Datomic test fixtures today)#2023-08-0517:18Dustin GetzUm, just noticed that file is also confused due to switching between datascript and datomic imports, i dont trust its integrity currently#2023-08-0517:19Dustin Getzit has to be meant for datomic though#2023-08-0509:32braai engineerWhy does RCF test unify on (d/pull db '[*] eid) but not on (->> eid (d/entity db) (d/touch)) ?
;; Touch should return hydrated entity, right?
(defn touch-entity [db eid]
(some-> (d/entity db eid) (d/touch)))
;; Pull should do the same, but annoying nil check:
(defn pull-entity [db eid]
(let [ent (d/pull db '[*] eid)]
(if (= {:db/id nil} ent)
nil
ent)))
"pull works"
(pull-entity db [:session/token token]) := {:db/id sid
:session/token token
:session/ticket {:db/id ticket-id}
:session/created-on _}
"touch fails. why?"
(touch-entity db [:session/token token]) := {:db/id sid
:session/token token
:session/ticket {:db/id ticket-id}
:session/created-on _}
#2023-08-0510:28Dustin Getzplease paste the test output#2023-08-0510:57braai engineer@U09K620SG see following post with details: https://clojurians.slack.com/archives/C7Q9GSHFV/p1691232765856069?thread_ts=1691226935.865749&cid=C7Q9GSHFV#2023-08-0510:58braai engineerOK, so this seems to be all related to (d/touch (d/entity db eid)) not working as I expect. Is this a type issue?#2023-08-0509:38braai engineerHow to check for thrown AssertionError with RCF? Like this?
(tests
(try
(do-broken-thing!)
(catch AssertionError ex
:except)) := :except) ;; ?
edit: Hmmk I see this in codebase https://github.com/hyperfiddle/rcf/blob/7105b43231140de6f2f39ce38611c9f6c9dfc976/src/hyperfiddle/rcf/impl.clj#L381-L392:
(do-broken-thing!) thrown? AssertionError
Suggest adding to README or docs.#2023-08-0518:26Dustin Getzit is :throws now actually, thrown? is legacy compat. I'll make a note to add it to the docs#2023-08-0518:40Dustin Getzdocumented#2023-08-0509:43joshchoI have a demo app that is based on electric-starter-app. It runs fine with clj -A:build -X user/main, but it fails for deployment via
HYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty`
clojure -X:build uberjar :jar-name "app.jar" :version '"'$HYPERFIDDLE_ELECTRIC_APP_VERSION'"'
java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -jar app.jar
Compilation runs fine, but in run-time it can't resolve certain symbol names. I am hosting this on a linode instance and accessing ip-address:8080. (But also runs into issues local).#2023-08-0509:43joshchoI can't quite pinpoint what's wrong. I have posted a public repo if anybody can help. Would be incredibly helpful! https://github.com/joshcho/push#2023-08-0509:48joshchoAlso, maybe this is because I'm using a cheap compute instance, but running with clj is incredibly slow. (Decent on mobile, slow on desktop for some reason.)#2023-08-0510:24Dustin Getzplease check your prod.clj specifically did you load all the namespaces #2023-08-0512:13braai engineerIs Missionary a suitable replacement for Promesa / futures?#2023-08-0512:16Dustin Getzyes#2023-08-0512:28leonoelMissionary provides tasks which cover the same problem space as futures with slightly different semantics. Futures are eager and memoized, tasks are lazy with optional memoization. Laziness is the key aspect allowing for transparent process supervision. I claim there is no use case for futures.#2023-08-0516:26joshcho2What is the best way to do optimistic updates that also sync with server? For instance, there is !server-number; when the user does inc, I want them to see the increased number (via client atom like !client-number) before we do a round trip of running (inc !server-number). I think I saw in the changelog that this may be the next thing on the list, but I am wondering what's the best way to do this currently.#2023-08-0516:32Dustin Getztoday you'd need to hold the state in a local atom, and run your server effect in response to the atom change#2023-08-0516:32Dustin Getztake care to rollback if you care to handle failure#2023-08-0516:39joshcho2I see, so iiuc, do you mean like a watch over a local atom? Then for multiplayer, would you make this bidirectional, i.e. watch over the server atom? (taking care not to introduce cycles)#2023-08-0516:47Dustin Getzessentially yes there is local state somewhere; in electric-ui4 (which i do not recommend you attempt to understand, we do not like it though it does work) there is a local atom which is very carefully controlled as the edits progress through various sync states#2023-08-0516:48Dustin Getzelectric-ui5 (not released) may model it differently – electric-ui5 is designed to have optimistic maintenance of a CollectionView, whereas ui4 only has optimistic individual fields#2023-08-0608:57joshcho2To get this done, is missionary necessary? Essentially, I want edits to client atom to be broadcast to server atom, and edits to server atom (not propagated by client) to broadcast to client atom. Been reading a lot of docs, but could not figure out how to do this exactly.#2023-08-0609:02joshcho2Also, how useful are docs like https://github.com/hyperfiddle/electric/blob/master/src-docs/wip/demo_todos_advanced.cljc? Been looking at that since it mentions optimistic updates.#2023-08-0707:26xificurCthe advanced demo is a POC that won't work as-is. It was basically our research demo to drive optimistic updates. The findings of this demo form the basis of ongoing internal rewrites that will unlock the option to model optimistic updates#2023-08-0711:26Dustin GetzYeah we've long moved past that implementation, removing the demos from that repo and putting the good ones into the tutorial is on my todo list#2023-08-0608:21braai engineerWhat's the newest JDK version I can build Electric with safely via Docker? Any benefits to newer versions, e.g. performance? Starter app Dockerfile uses OpenJDK 11 and runs with Amazon Corretto 11: https://github.com/hyperfiddle/electric-starter-app/blob/main/Dockerfile#2023-08-0608:28braai engineerElectric app runs locally via REPL & builds via Docker but when I try to build uberjar on host or via Docker, I get this exception about a Protocol:
Exception in thread "main" java.lang.NoClassDefFoundError: myproject/proto/AMyProto
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
...
Caused by: java.lang.ClassNotFoundException: myproject.proto.AMyProto
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
... 114 more
prod.clj has (:gen-class)
another gen-class issue? AOT?
;; proto:
(defprotocol AMyProto
(do-thing [this ...])
...)
;; mock file:
(ns mymock
(:require [myproject.proto :as proto]))
(defrecord MyImpl [!history secret-token] ;; access token?
proto/AMyProto
(do-thing [this ...] ...))
Hmm, any ideas? Could it be JDK version issue?#2023-08-0608:37braai engineerTried adding (:gen-class) to myproject.proto namespace, but no luck.#2023-08-0608:39braai engineeralso happens on host machine that's running Java Oracle 20.0.01#2023-08-0608:42braai engineerRequiring myproject.proto NS from prod.clj does not help#2023-08-0612:20Dustin Getzthis is not an electric issue#2023-08-0612:20Dustin Getzdid you resolve it?#2023-08-0612:21braai engineerNo 😕 Temporary workaround by deploying dev build to prod#2023-08-0612:21Dustin Getzthis is likely the root problem causing the rest of your problems#2023-08-0612:21braai engineermy pain is no logs#2023-08-0612:21braai engineerjust reactorfailure#2023-08-0612:22Dustin Getzlogs are not broken, please check out a clean starter app and convince yourself they work#2023-08-0612:22Dustin Getzyou can add more logs to your app's boot sequence#2023-08-0612:22braai engineerI compared mine to the starter-app and seems to be same#2023-08-0612:22Dustin Getzyou can also turn log level up to trace#2023-08-0612:22braai engineerok will try that#2023-08-0612:22braai engineermaybe I need to catch exception in jetty side#2023-08-0612:22Dustin Getzi want to be clear: if logs are broken, it is your fault#2023-08-0612:23braai engineersure, but I've had logging issues before so I'd rather mention it than be stuck for hours#2023-08-0612:23Dustin GetzElectric Does Not Have Logging Issues, if you suspect otherwise please checkout a clean starter app and confirm that it is true, and save yourself the hours of pain debugging something that is not broken#2023-08-0612:53braai engineerok - just trying to get things working 🙂#2023-08-0608:42braai engineerHow should I prevent deployment if RCF tests fail? Does someone have a Github Actions or Dokku template lying around?#2023-08-0612:26Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/.github/workflows/test_and_deploy.yml#2023-08-0609:39braai engineerReactorFailure on prod uberjar build, but works fine in REPL build. No errors in server logs - only client.
Connecting...
main.9BFB58C0E5DF4F15F4C87A7ED5236453.js:1522 Connected.
main.9BFB58C0E5DF4F15F4C87A7ED5236453.js:1419 WebSocket is already in CLOSING or CLOSED state.
Reactor failure:
"Remote error - 1011 org.eclipse.jetty.websocket.core.exception.WebSocketException:
Error: Remote error - 1011 org.eclipse.jetty.websocket.core.exception.WebSocketException: WebSocketAdapter$WebSocketPingPongListener$12d400b6 OPEN met
at new Tm ()
at Wm ()
at Vm ()
at p ()
at Function.t [as o] ()
at lT ()
at e.resume ()
at lT ()
at e.resume ()
at mU ()"
~This is probably a Datomic error due to lack of schema or seed data~, but why zero logs on server? Verified it's not Datomic. I am transacting schema and can run queries via an API endpoint.
I compared my logback.xml to starter app settings and don't see an issue.
plz halp deploy blocked#2023-08-0609:59braai engineerlogback.xml:
<!-- Jetty logger config. See and
ask ChatGPT. `scan` and `scanPeriod` enable hot reload of logger config, leave on! -->
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%highlight(%-5level) %logger: %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="STDOUT" />
</root>
<logger name="hyperfiddle" level="DEBUG" additivity="false"><appender-ref ref="STDOUT" /></logger>
<!-- <logger name="hyperfiddle.electric.impl.io" level="INFO" additivity="false"><appender-ref ref="STDOUT" /></logger> -->
<logger name="quey" level="DEBUG" additivity="false"><appender-ref ref="STDOUT" /></logger>
</configuration>#2023-08-0609:59braai engineer@U09K620SG I've added an nREPL into my uberjar build. How do I trigger an Electric run from REPL to try and see an error, please? (e/run something)?#2023-08-0610:00braai engineerthis doesn't seem to work:
(ns scratch
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]))
(e/defn Test []
(dom/div "hi"))
(e/run (Test.))#2023-08-0610:07braai engineerOK this runs "in prod" but obviously fails because DOM stuff missing on server for my components:
(ns scratch
(:require [hyperfiddle.rcf :as rcf :refer (tests tap %)]
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[missionary.core :as m]
[hyperfiddle.history :as history];; for core
[quey.manage.core :as core]
[datomic.api :as d]))
(e/defn SanityTest [!x]
(prn "hi")
(let [x (e/watch !x) ; reactive x
a (inc x)] ; reactive a depends on reactive x
(tap (+ a (inc x)))))
;(d/q nil nil))
;(e/run Test)
(tests "Electric baseline program - a dataflow diamond"
(def dispose (e/run (core/Page. [:thing]))) ;; fails because DOM stuff
;; rest is jsut here to test
(def !x (atom 0))
(def dispose (e/run (SanityTest. !x)))
% := 2
(swap! !x inc)
% := 4
(swap! !x inc)
% := 6
(dispose))#2023-08-0610:41braai engineerI had to resort to deploying a dev build because I can't get uberjar working (ReactorFailure - zero server logs). How can I hide the shadow-cljs thingy until I can fix?#2023-08-0618:25joshchoI am trying to get only the first value of a flow. I am currently doing something like this:
(->> (e/fn [] (e/server (:task/toggled (d/entity db task-id))))
(m/eduction (filter boolean?))
(m/reductions conj [])
new
first)
This feels roundabout, as the rest of the flow still is received, only to be discarded (maybe that's okay). Is there a better way to do this?#2023-08-0618:29Dustin Getze/snapshot ?#2023-08-0618:52Dustin Getzuse e/watch to bridge atoms to electric, which you already know, i think you might be over thinking it#2023-08-0618:53Dustin Getz(e/server (println (e/client (e/watch !x)))) accomplishes your goal of running an effect on the server every time an atom changes#2023-08-0618:54Dustin Getzoops, did you delete a question? or somehow i replied to wrong thread#2023-08-0618:58joshchoAh, no. I ended up figuring it out! I definitely was overthinking it.#2023-08-0619:01joshchoThe incremental compiler model is very powerful, I just need to get more in the habit of thinking that way.#2023-08-0703:07JAtkinsCan the electric compiler be described as decomposing an app into continuations? With better caching of unchanged values ofc. If this is an accurate understanding it may help others too. #2023-08-0707:47xificurCcan you elaborate on what it means for you?#2023-08-0712:36JAtkinsIgnoring the network transfer, an e/watch on an atom could be recreated with a continuation. #2023-08-0704:40joshchoIs there a better way to do optimistic updating from client to server, but also server to client propagation? Managed to put together https://gist.github.com/joshcho/3c0ec077143dd15231b983042d4c2dc9 using only Electric (and no Missionary), but it's not too perfect.#2023-08-0707:52xificurCthe ui4 namespace contains controls with optimistic updates#2023-08-0713:25joshchoRight, I am looking at ui/control. I think my use case is a little different, as I want to expose the atom itself. I am trying to use the code in ui/control as an example, but not really succeeding. Specifically, I want to change listen> call to (e/fn [] (e/watch !client-value)), but it's not working.#2023-08-0713:35xificurCto clarify what we're talking about can you state the use case? The initial, concrete one#2023-08-0714:06joshchoSure, I have a textarea corresponding to (:task/notes (d/entity db task-id)) on the server. Previously, I had a watch over that query and on the change of textarea, I would update over the wire via something like:
(e/server (d/transact! !conn [{:db/id task-id :task/notes task-notes}]))
Then the user sees the updated textarea when the round-trip ends, which is not ideal. I want the user to see the changes right away, but still have the changes sent over the wire.
In addition, I want this to be multiplayer, so if another user makes changes, I want the changes to be propagated back to the original user. Maybe timestamps could resolve conflicts.
I think I may be oversimplifying a concurrency problem. I have been trying with the above gist (which goes into an infinite cycle) and missionary (which I am trying to figure out atm).#2023-08-0714:11joshchoWait. I think when I tested ui/textarea, I didn't do it properly. Does ui/textarea already do this?#2023-08-0714:13xificurCindeed, ui/textarea does what you described#2023-08-0714:13joshchoPerfect, ty.#2023-08-0714:25joshchoAh, I remembered the use case that had me wondering this in the first place. I have a toggle button, which looks like this:
(e/defn Toggle [task-id !toggled]
(let [toggled (e/watch !toggled)]
(ui/button
(e/fn []
(swap! !toggled not))
(svg/svg (dom/props {:width 10
:height 10})
(when toggled
(SVGVerticalLine.))
(SVGHorizontalLine.)))))
Toggle takes in !toggled because the caller of Toggle uses !toggled to change the visibility of subitems:
(let [task (e/server
(:task/name (d/entity db task-id)))
subtask-ids (e/server
(sort (map :db/id (:task/subtask (d/entity db task-id)))))
!toggled (atom nil)]
;; (make-relay !toggled
;; (:task/toggled (d/entity db task-id))
;; nil
;; (fn [v]
;; (tx/transact!
;; !conn [{:db/id task-id :task/toggled v}]))
;; boolean?)
(dom/div
(ui/button
(e/fn []
;; for disabling double-click
(if (= selected-id task-id)
(when-not (= selected-id running-id)
(run-selected-task.))
(reset! !selected-id task-id)))
(dom/div
(dom/props {:class (when-not (and (seq subtask-ids)
(not editing))
"hidden")})
(Toggle. task-id !toggled))))
(dom/div
(dom/props {:class "ml-2"})
(dom/div
(dom/props {:class
(when (e/watch !toggled)
"hidden")})
(TaskList. subtask-ids))))
The commented out make-relay is what I had previously to make !toggled into a relay atom.#2023-08-0714:33xificurCwhy does Toggle take a task-id? It isn't used in the snippet#2023-08-0714:36xificurCwhen the DOM tree aligns nicely you can hide the atom completely
(e/defn Toggle []
(let [!toggled (atom false)]
(ui/button ...)
(e/watch !toggled))
Now it returns its state as a value. Since this mounts the button and returns a value you often run into DOM position issues where you need to have an atom on the outside#2023-08-0714:57joshchoThat makes sense. What if you wanted to make toggle state persistent in the server? Then I would need something like this:
(e/defn Toggle [task-id]
(let [toggled (e/server (:task/toggled (d/entity db task-id)))]
(ui/button
(e/fn []
(tx/transact!
!conn [{:db/id task-id :task/toggled v}]))
(dom/props {:class (tw "ml-1 btn-[* xs] w-fit px-1")})
(dom/on "click" (e/fn [e]
(.stopPropagation e)))
(svg/svg (dom/props {:width 10
:height 10})
(when toggled
(SVGVerticalLine.))
(SVGHorizontalLine.)))))
But somehow expose toggled state to its caller. Also, unlike textarea, there would be a delay since toggled wouldn't update until we do the round trip. I think ui/checkbox wouldn't work either since it doesn't expose the optimistically updated dom value.#2023-08-0714:57joshchoAlso, thank you for answering all these questions, it's been tough trying to get this working#2023-08-0715:05xificurCwhat's v in this example? Should it be (not toggled)?#2023-08-0715:06joshchoYeah, sorry, I am sort of patching this code for example.#2023-08-0715:07joshchoLet me revert and give you something that I was using#2023-08-0715:09joshchoThis was what I had in full before optimistic rendering, and I wanted to remove the latency:
(e/defn Toggle [task-id]
(let [toggled (if (u/in? temp-show-task-ids task-id)
false
(e/server (:task/toggled (d/entity db task-id))))]
(ui/button
(e/fn []
(e/server
(tx/transact! !conn [{:db/id task-id
:task/toggled
(not (if (some #(= % task-id) (e/client
@!temp-show-task-ids))
false
(:task/toggled (d/entity @!conn task-id))))}]))
(reset! !temp-show-task-ids nil))
(dom/props {:class (tw "ml-1 btn-[* xs] w-fit px-1")})
(dom/on "click" (e/fn [e]
(.stopPropagation e)))
(svg/svg (dom/props {:width 10
:height 10})
(when toggled
(SVGVerticalLine.))
(SVGHorizontalLine.)))))
(e/defn TasksPanel []
(dom/div
(dom/props {:class "grow sm:max-w-xs min-h-[14rem] sm:min-h-[18rem] min-h-full sm:h-none mb-2 sm:mb-0 p-2 rounded bg-base-200"})
(dom/on "click"
(e/fn [e]
(e/server (reset! !selected-id nil))))
(let [!editing (atom false), editing (e/watch !editing)]
(dom/div
(dom/props {:class "text-xl font-bold flex items-center"})
(ui/button
(e/fn [] (e/server (reset! !selected-id running-id)))
(dom/on "click" (e/fn [e]
(.stopPropagation e)))
(dom/text "Day 5"))
(ui/button
(e/fn [] (swap! !editing not))
(dom/props {:class (tw "ml-2 btn-[* xs] mt-[1px] bg-base-100")})
(dom/on "click" (e/fn [e]
(.stopPropagation e)))
(dom/text (if editing "stop editing" "edit"))))
(dom/div
(binding
[TaskList
(e/fn [task-ids]
(e/for [[is-last task-id]
(map-indexed (fn [idx task-id]
[(= idx (dec (count task-ids))) task-id])
task-ids)]
(let [task (e/server
(:task/name (d/entity db task-id)))
subtask-ids (e/server
(sort (map :db/id (:task/subtask (d/entity db task-id)))))]
(dom/div
(dom/props
{:class (tw "flex"
;; don't use cond
(when (= task-id running-id)
"font-bold"))})
(ui/button
(e/fn []
;; for disabling double-click
;; (e/server (reset! !selected-id task-id))
(if (= selected-id task-id)
(when-not (= selected-id running-id)
(e/server (run-selected-task)))
(e/server (reset! !selected-id task-id))))
(dom/props {:class "text-left flex"})
(dom/on "click"
(e/fn [e]
(.stopPropagation e)))
(dom/div
(dom/props {:class (when (= task-id selected-id)
"underline")})
(dom/text task))
(dom/div
(dom/props {:class (when-not (and (seq subtask-ids)
(not editing))
"hidden")})
(Toggle. task-id)))
(dom/div
(dom/props {:class (when-not editing
"hidden")})
(TaskActionButtons. task-id)))
(dom/div
(dom/props {:class "ml-2"})
(dom/div
(dom/props {:class
(when (if (u/in? temp-show-task-ids
task-id)
false
(e/server
(:task/toggled
(d/entity db task-id))))
"hidden")})
(TaskList. subtask-ids))))))]
(dom/props {:class "mt-[2px]"})
(TaskList. (e/server (db/get-root-task-ids db))))))))#2023-08-0715:36xificurCI'll think about it a bit more and get back to you. We haven't fully tackled optimistic updates and there's internal rewrites that will help unleashing new patterns. For now we have to resort to simpler solutions.
The dumbest thing you can do and might work reasonably is set the value on the button. You'd have (set! (.-task-toggled dom/node) toggled) for the server value and (set! (.-task-toggled dom/node) (not (.-task-toggled dom/node))) on each click. That way the local value will be the optimistic one until a new server value arrives#2023-08-0715:44joshchoThank you! Looking forward to the next release. My simple solution for now works for things like toggle, and for input I can just use ui/input, so we are good.#2023-08-0707:39henrikRather than side-effecting in JS events, I’m looking at the pattern of using the return value.
So let’s say I have,
(reset! some-atom (dom/on! "input" dh/target-value))
Where dh/target-value is just a wrapper for the equivalent JS fields.
What’s not great is that dom/on! seems to instantiate with nil. I don’t want to run reset! at all until a value has materialized.
So I tried the following:
(defn nil->pending
[flow]
(m/reductions (fn [_ y]
(if (nil? y)
(Failure. (Pending.))
y))
nil
flow))
(e/defn Nil->Pending
[value]
(new (nil->pending (e/fn [] value))))
(reset! some-atom (Nil->Pending. (dom/on! "input" dh/target-value)))
This appears to work. But is it sensible? Good?
Oh, and I realize a when-some could do the same work here. This is 1) more out of interest to understand Electric than to arrive at the simplest solution and, 2) If it is sensible, I can make specialized on-wrappers that default to Pending for these situations, and skip the when-some boilerplate.#2023-08-0708:10xificurCThis will return the current DOM node's input value as a continuous value. Is that what you were looking for? Happy to explain the parts of this little machine if you want#2023-08-0708:26henrikOh yeah, please do!
In the snippet, are we throwing away the target value? I would consider that incidental, and that I might be looking for any value on the event.#2023-08-0708:27henrikAh, no, you’re just defaulting to the value of the node with reductions, right?#2023-08-0708:30henrikThe atom already contains a value that is not to be replaced with anything until the event is fired by user action. I.e., the node value is not going to be correct out of the gate.
So then I could make the default be the value already present in the atom. But again, I would consider that to be specialized for the example. The general effect I would like to achieve is to not run reset! (or whatever code might be there) until there’s a user action, as would be the case when side-effecting the same thing.#2023-08-0708:49xificurCisn't that dom/on! then?
(dom/on! "input" (fn [e] (reset! some-atom (dh/target-value e))))
#2023-08-0709:08henrikAs far as I can see, it returns nil on mount.#2023-08-0709:09henrikI mean, yes, side-effecting solves the entire thing. The objective is to not use side-effects.#2023-08-0709:12henrikOr otherwise learn something: why it shouldn’t/can’t be done that way.#2023-08-0709:18xificurCI lost track of the requirements. I thought you're starting with an atom and want to update that atom's value on every input change#2023-08-0709:21henrikMore generally, I’m trying to go from side-effecting in events to using a functional model instead: using the return value.
In order to achieve the same behavior as when side-effecting, I have to get rid of the initial value. Otherwise they diverge like this:
Side-effect: meaningful-value-1, meaningful-value-2, …
Functional: nil, meaningful-value-1, meaningful-value-2, …#2023-08-0709:25henrikThe atom in the example is just an example, and none of this is strictly necessary since I can accomplish what I want by side-effecting.
However, I like the feeling of tight loops to external things that Electric provides. Without much intermediary fuss, I’m back in the graph when working with router events, window resizing, etc. No need to land values in intermediate state etc.
So this is kind of just chasing that feeling, but for events.#2023-08-0709:31xificurCI think what you're saying is you're unsure how to model discrete effects as a continuous time value since the latter requires an initial value. You can model the lack of an initial value with Pending, in which case you get something like this#2023-08-0709:34henrikAh nice. That looks close to what I have, but with fewer steps.#2023-08-0709:35xificurCthese are tricky things, though, the semantics change in slight ways. E.g. in continuous time we only care about the latest value, so it's ok to skip intermediate steps, which means with 2 quick reset!s the first one might not run in CT#2023-08-0709:35henrikHence relieve baked in?#2023-08-0709:36henrikOr is this problem still there in spite of relieve?#2023-08-0709:36xificurC(m/latest #(reset! ..) flow) , here some reset!s might not run#2023-08-0709:37henrikLet’s assume that skipping some events is fine, the latest value is the only relevant one.#2023-08-0709:38xificurCsure, feel free to try the above out#2023-08-0709:42xificurCe/listen> already relieves so I don't think it's needed later, therefore I removed it from the above sample#2023-08-0710:56henrikI’m not quite there with this. For some reason, the nil is omitted (good), but all other events are omitted too (awkward). dh/target-value never runs.#2023-08-0711:01xificurCcan you share a minimal repro I can run?#2023-08-0711:02henrik;; Doesn't run
(dom/input
(new (m/reductions {} (hyperfiddle.electric.Failure. (hyperfiddle.electric.Pending.))
(e/listen> dom/node "input" (fn [event]
(println event))))))
;; Does run
(dom/input
(new (m/reductions {} nil
(e/listen> dom/node "input" (fn [event]
(println event))))))#2023-08-0711:07henrikOTOH,
(->> (e/listen> dom/node "input" (fn [event] (println event)))
(m/reductions x/discard)
new)
seems to be exhibiting the correct behavior without a default value.#2023-08-0711:10henrikNo, false alarm. But,
(let [res (new (m/reductions (fn [_ y]
(println "y:" y)
(if (nil? y)
(hyperfiddle.electric.Failure. (hyperfiddle.electric.Pending.))
y))
(e/listen> dom/node "input" dh/target-value)))]
(println "res:" res)
res)
prints y:, but not res: on mount.#2023-08-0711:13henrikYeah, I think this last piece does what I’m aiming for. But fail/pending in reductions perhaps cancels a bit pre-emptively when stuck right inside a e/defn?#2023-08-0711:20xificurCyeah, sticking Failure in electric doesn't work. Put the part before new in a helper clj fn, e.g.
(defn my-helper [node typ] (m/reductions {} (Failure. (Pending.)) (e/listen> node typ dh/target-value)))
and (new (my-helper dom/node "input"))#2023-08-0711:23henrik👍
(defn on-and-on-and-on
[node type f]
(m/reductions {} (hyperfiddle.electric.Failure. (hyperfiddle.electric.Pending.))
(e/listen> node type f)))
Seems OK#2023-08-0711:26xificurCto elaborate a bit - electric code (throw (Exception.)) turns into a (Failure. (Exception.)) for missionary processing. So if you're modeling something exceptional for electric in missionary you use Failure#2023-08-0711:27henrikAh, gotcha. That something is “pending” doesn’t seem like an exceptional situation, but a common one. But I get the gist.#2023-08-0711:29xificurCexceptions in a reactive language are a different beast, so it's not exactly apples to apples. One way to think of them is they are an implicit Either monad and we present a missing value (pending) with the error part of that monad#2023-08-0711:30henrikThe try/catch mechanism matches the desired behavior, but the nomenclature is a bit off, I take it.#2023-08-0711:34henrikOr rather, debating whether “pending” is an “exception” as in exceptional vs. an “exception” as in the programming construct is semantics.#2023-08-0711:36henrikAnyway, the above works, so thanks very much for bearing with me.#2023-08-0711:37henrikFor posterity, here’s the code I’ve ended up with:
(defn on*
[node type f]
#?(:cljs (m/reductions {} (Failure. (Pending.))
(e/listen> node type f))))
(defmacro on!
[type f]
`(new (on* dom/node ~type ~f)))#2023-08-0711:39henrikI’ll find a better name for the macro later. off! maybe.#2023-08-0711:43xificurCnp 😉 A side note - the function passed to e/listen> is a keep fn, i.e. it won't propagate nil outwards#2023-08-0711:45henrikBrilliant!
I’m left with the thought that it might be better for the behavior we have here to be default for on!, since introducing the nil is just a matter of try-catch-Pending, while going the other way turns out to be quite an adventure.#2023-08-0712:52Dustin GetzTo address broadly the idea of returning events as values – you're right and this is the future path forward, dom/on has some problems e.g.:
• https://clojurians.slack.com/archives/C7Q9GSHFV/p1687539632604829?thread_ts=1687533303.028249&cid=C7Q9GSHFV
• https://clojurians.slack.com/archives/C7Q9GSHFV/p1684443621877529?thread_ts=1684442493.333779&cid=C7Q9GSHFV
• https://clojurians.slack.com/archives/C7Q9GSHFV/p1686248211536109
The React-style change callbacks was just a way for us all to make forward progress while we figure it out – perfecting the signals-based design is driving improvements to electric itself. To be clear, dom/on works fine and doesn't have bugs that we are aware of, you should use it until we provide new guidance#2023-08-0712:55henrikThe UI stuff comes with assumptions (notably: styling) that won’t work for me.#2023-08-0712:55Dustin Getzadded 3rd link#2023-08-0712:56Dustin GetzUnderstood – the UI controls are intended to be general, we will find a way to separate out the style/markup into a wrapper or something#2023-08-0712:56Dustin Getzit's all very WIP currently#2023-08-0713:05henrikYeah, completely understood, and I took it as such.
Of all places for inspiration, I think React has the right idea re: decomplecting when they went to functional components. They observed that the stateful components that emerged from the class-based stuff were causing problems (notably, a proliferation of parochial APIs and deep component hierarchies, though I’m framing this in Clojure lingo). Hooks vs. components.
To me, it looks a lot like solving the Expression Problem—same reason we have protocols and multimethods. You put behavior to one side, and presentation/structure to the other, and then you compose them.
If you pick a set of components, then apply a set of behaviors to them, the problem doesn’t vanish, but it shrinks considerably.
If you could approach the effort of building reusable UI components with that frame of mind, I think you’ll get a more general solution.#2023-08-0713:08henrikIn particular, for some of the more tricky stuff we’ve been building, all prepackaged component libraries (where behavior and structure is complected) have failed us. What has worked has been libraries that provide hooks, which we then compose using a mix of our own and OOTB components.#2023-08-0713:18henrikSorry if I’m preaching to the choir (or sound preachy at least), it sprang to mind when looking at the UI NS.#2023-08-0713:44Dustin GetzYeah, basically our strategy is to delay abstraction until a commercial project presents us with concrete requirements to implement. Having been bitten by the "parochial" patterns you mention. UI4 is hardcoded to a specific experimental project#2023-08-0713:45Dustin GetzIt's not actually clear to me if there is a general solution to this as the DOM directly couples ~implementation and layout~ *structure and markup/presentation#2023-08-0713:46Dustin GetzElectric at least can render structure into a container that is controlled by the caller, rather than hardcoding divs and such#2023-08-0713:47Dustin GetzYou mention a "hooks" based pattern that worked for you, can you link the libs you use?#2023-08-0714:19Dustin Getzoh i see, we are saying the same thing#2023-08-0714:20Dustin Getzi still would like to see the hooks but you mean a pattern where widget internals are unbundled and assembled by the app developer in a way that the dev controls all markup#2023-08-0714:34henrikNot only markup, but also where behavior applies. With the command palette/menu for example, rather than providing a set of “components” into which you feed data, I emit functions that you call on event handlers and what not from within whatever components you want to use. This means,
• No tight bundling between the behavior of a menu and its structure when displayed.
• You can still apply the behaviors to a set of components and have an out-of-the-box solution that might work 70% of the time that you need a menu. When it doesn’t, the behaviors are accessible from “user space”.
• The “logic” of the menu is technically platform independent, although this is not a primary concern right now. But technically, it could provide the same behavior to ClojureDart, for example, theoretically.
• Generally leads to better architecture (or separation of concerns, at least). Things that are separate to the “logic” of a menu are clearly outside of it, such as when and how it scrolls if space is constrained, how it displays on smaller screen sizes etc. #2023-08-0714:39henrikIf you go looking, most menu libs are all over the place with these things, with an unclear definition of what the core behaviors of a menu are. Since these aspects are not clearly separated, they tend to have an ever growing API driven by feature requests for corner cases where some assumption in the interplay between behavior and structure didn’t hold up for some particular use case.#2023-08-0714:41henrikhttps://headlessui.com/ has the right idea that presentation should be left to the user, but they still provide you with components. This lib worked up to a point for us, and then fell apart.#2023-08-0714:42henrikhttps://floating-ui.com/docs/react has a much better idea of how to break the problem apart.#2023-08-0714:46henrikBtw., I don’t think much of this applies to low-level stuff like how an input works. It generally comes into play when the component exhibits a bit more complexity by its nature.#2023-08-0714:47Dustin Getzright, ok. thanks#2023-08-0714:49henrikLet’s say a structured input, where you insert dashes or spaces when typing phone numbers. Bundling the behavior with input is silly, but a phone-number-input is equally silly. A function or set of functions that you can apply on an input, or several inputs, depending on the implementation would be the way to go IMO.#2023-08-0714:28braai engineerLooking at Electric network traffic, I see a lot of WS frames for a button click that logs a message on server and does not return anything.
I'm assuming a lot of this can be optimized away in future?
(dom/button
(dom/on "click"
(e/fn [e]
(e/server
(prn "nothing")
nil)))
(dom/text "click me"))#2023-08-0714:30Dustin GetzThe traffic is not what you assume it is, see https://news.ycombinator.com/item?id=36877238#2023-08-0714:32Dustin Getz1. yes, it will be optimized
2. this is "UDP" "broadcast" messages, not RPC or round trips, therefore does NOT introduce latency as the client is not awaiting any response
3. the mobile radio is already on, so the battery impact is insignificant#2023-08-0714:32Dustin GetzMost importantly, dom/on is simply not an efficient pattern and we are replacing it, at which point most of this traffic will evaporate. Problem is dom/on rapidly mounts and unmounts an electric lambda, causing DAG maintenance overhead, which is the traffic you see. To be clear, dom/on works fine, and is not laggy nor buggy and you should use it, we merely think there is a better pattern which we have not released yet.#2023-08-0714:37Dustin Getzalso, the traffic log might seem to suggest that there is back and forth chatter. It's not so; the log is interleaving upload and download traffic occurring simultaneously but that does NOT mean there is causal dependency between each message#2023-08-0714:39Dustin Getzfinally, transit (the serialization protocol) is in verbose mode which is heavy. We have a binary protocol which is both lighter in payload size and lighter to parse. It's currently turned off, I forget why#2023-08-0714:41Dustin Getzif i'm not mistaken, the screenshot shows one long upload burst (green) and one shorter download burst (red) occurring simultaneously. The line records shouldn't be seen as independent, it's a stream, we burst a bunch of lines that are considered simultaneous, and then its done#2023-08-0714:41Dustin Getzlikely the server finished running some effect before the client even finished uploading#2023-08-0714:43braai engineerTesting electric-starter-app on my phone (using Firefox): if I lock my phone after looking at starter app and later reopen it, the view still displays and I can tick items, but it no longer syncs with server. I have to refresh the view to reconnect the WS.
How can I tell if the client has server connectivity?#2023-08-0714:44Dustin Getzthat sounds like a bug, we fixed a heartbeat issue in master recently that will be released soon#2023-08-0714:45braai engineerwould be nice if there was a dynamic binding I could check for connectivity so I could render a "Disconnected" message (edit: less caps)#2023-08-0714:46Dustin Getz• the expected behavior currently is: client will gracefully shutdown on connection loss due, then gracefully reconnect when the tab is restored
• future behavior: client will stay alive throughout the disconnection window and gracefully reconnect without loss of any state, queueing messages while offline and flushing them through on reconnect#2023-08-0721:07braai engineerIs there an example of having two Electric builds / reactors in one project?
I want a "thin" Electric client for mobile users on the frontend that doesn't load all the admin stuff.
I assume I can reuse the same server but serve a different set of modules?#2023-08-0721:46Dustin Getzi dont know of anyone who has tried this yet, i think it should work#2023-08-0721:47Dustin Getzare you trying to work around a specific problem?#2023-08-0721:47Dustin Getze.g. long compile times?#2023-08-0721:51braai engineermobile data is expensive in ZA (we pay per MB) + loading times. I understand that client loads/sends whole graph to server?
I want to minimize the data transfer for a customer page but still have power of electric#2023-08-0721:53Dustin Getzdo you have measurements?#2023-08-0721:53braai engineerI haven't measured no#2023-08-0721:54braai engineermain.js is 1.2MB#2023-08-0721:54braai engineerthat is a lot, but OK that's a dev build#2023-08-0721:54Dustin Getzis that compressed?#2023-08-0721:55Dustin GetzIt may be, the macroexpansion weight has not been considered, we know its bad#2023-08-0721:55braai engineertrying reagent for now#2023-08-0721:55braai engineergot some shadow-cljs port issues clashing with Electric - will post new thread#2023-08-0721:55braai engineerneed to ship something for tomorrow#2023-08-0721:56braai engineershadow-cljs port-clash: This is probably shadow-cljs related but I can't figure out how to get Electric shadow and my (other) frontend shadow-cljs build to play along despite changing nrepl port and :http port:
Starting Electric compiler and server...
Execution error (ExceptionInfo) at shadow.cljs.devtools.server/check-for-other-instance! (server.clj:415).
shadow-cljs already running in project on . Use or terminate it before starting another one.
shadow-cljs.edn looks like this:
{:deps {:aliases [:dev]}
:nrepl {:port 9001}
:builds
{:dev {:target :browser
:devtools {:watch-dir "resources/public" ; live reload CSS
:hud #{:errors :progress}
:ignore-warnings true} ; warnings don't prevent hot-reload
:output-dir "resources/public/js"
:asset-path "/js"
:modules {:main {:entries [user]
:init-fn user/start!}}
:build-hooks [(shadow.cljs.build-report/hook {:output-to "target/build_report.html"})
(user/rcf-shadow-hook)]}
:prod {:target :browser
:output-dir "resources/public/js"
:asset-path "/js"
:module-hash-names true
:modules {:main {:entries [user] :init-fn user/start!}}}
:frontend {:target :browser
:output-dir "resources/public/js-frontend"
:asset-path "/js-frontend"
:module-hash-names true
:http {:port 9650}
:nrepl {:port 9631}
:compiler-options {:externs ["datascript/externs.js"]}
:modules {:main {:entries [myproject.frontend.core]
:init-fn myproject.frontend.core/init}}}}}#2023-08-0722:09Dustin Getzmust they run concurrently?#2023-08-0722:11Dustin Getzi think you can use the same shadow instance also, if you control shadow-cljs compilation from the REPL API you can pass in the build-id#2023-08-0722:11Dustin Getzhttps://github.com/hyperfiddle/electric-starter-app/blob/82a6006081935c55f944b717bd9ec4f88731d1c7/src/user.clj#L14#2023-08-0810:14braai engineerFor dev it is convenient for them to run concurrently. I guess I could even start my thin frontend shadow-cljs build from user.clj#2023-08-0721:56telekidLess than a month ago, I began learning Electric, and then started using it to build the UI for my project, which I'm looking to raise money for shortly. (I started building out the backend a few months prior.) Lo and behold, I'm finished with the prototype and have put together https://www.youtube.com/watch?v=mGBo8eibTR8 for investors. I'm posting here so that people can get an idea of the absolutely insane level of productivity Electric affords. It would have been really challenging to achieve this level of interactivity without Electric. (Please don't distribute the video outside of this channel, I'm still in "single developer stealth mode," though I'm not taking that too seriously.)#2023-08-0722:07Dustin Getzimpressive demo, congrats!!!#2023-08-0722:25telekidThank you!#2023-08-0803:43joshchoamazing#2023-08-0807:29henrikVery cool! Best of luck with the investors.#2023-08-0723:37markaddlemanhttps://opensource-heroes.com/discover/clojure#2023-08-0723:38markaddlemanElectric and three electric-related projects on the top 100!#2023-08-0812:31joshchoWould a e/binding macro be useful that creates bindings for both client and server scope be useful? Something like this:
(defmacro sync-binding
"Does binding in both server and client scope. Make sure to explicitly specify the scope in bindings (e.g. e/client or e/server). Body is evaluated in client scope."
;; possibly improve by preserving the caller's scope
{:style/indent 1}
[bindings & body]
`(e/server
(binding ~bindings
(e/client
(binding ~bindings
~@body)))))
One nice thing with Electric is since lexical variables are automatically transferred, I don't have to nitpicky about where the variable is coming from, and maybe we can preserve those semantics for binding.#2023-08-0812:35Dustin Getzmaybe, unclear – i was playing with something like this too (focused on pyramid flattening), see https://github.com/hyperfiddle/electric/blob/2dc4f9a1b555df343e646452e3045da04a84344a/src/contrib/clojurex.cljc#L14-L20#2023-08-0812:36Dustin Getzrequirements seem to be:
• no pyramids (allow causal dependencies in bindings)
• e/client and e/server at any point, strategy for dealing with it
• support both let and binding (perhaps based on presence of a var? perhaps too error prone)
#2023-08-0812:37Dustin Getzyou will eventually hit some complexities about clojure bindings vs electric bindings that is on our backlog to take a fresh look at#2023-08-0812:38Dustin Getzalso be aware of the issue "no electric binding unification yet" – when you e/def, you get both clojurescript and clojure electric def – there are two defs – which is why you need to be super explicit when you read an electric binding (as opposed to reading let bindings which are always bound to a single site) . This also is on our backlog to fix this fall#2023-08-0812:40joshchoRight. Maybe things can get real thorny if the two defs go out of sync.#2023-08-0816:12henrikI’ve been wanting something like this. I have a “boot” section of the app where server and client blocks are taking turns throwing stuff at each other so that a bag of e/defs end up synchonizing.
It got trickier when some bindings became inputs to other bindings. I briefly thought of throwing Ubergraph at it to resolve dependency order behind a macro.#2023-08-0920:25Dustin Getz@U06B8J0AJ bindx above should help with the dependency issue, however I never got it working in clojurescript and have no idea currently why it doesn't work in clojurescript#2023-08-0920:26Dustin Getznote the funcool/cats alet macro includes topo sorting capability over alet bindings, you could probably copy paste it subject to their license#2023-08-0814:35joshchoIn the following code, is an infinite cycle expected when pressing the button? It's a bit weird to me since I would expect the e/fn to be executed only once. Perhaps this is because of the function-component duality.
(defonce !conn (d/create-conn {}))
(e/def some-value (e/server (:value (d/entity (e/watch !conn) 1000))))
(e/defn App []
(e/client
(e/server (d/transact! !conn [{:db/id 1000 :value 10}]))
(ui/button
(e/fn []
(e/server
(println "button pressed") ; prints forever
(d/transact! !conn [{:db/id 1000 :value (+ some-value 1)}])))
(dom/text "[Increase Value by 1]"))
(dom/text some-value)))#2023-08-0814:41joshchoFor this particular case, the fix is easy (just change some-value to (:value (d/entity @!conn) 1000) , but for my actual use case, I have several reactively connected e/def that are used within e/fn. It would be really handy to be able to write like functions like this because otherwise, I wouldn't be able to reuse any logic within these reactive atoms.#2023-08-0814:45xificurCthis is a more complicated example of
(let [!x (atom 0), x (e/watch !x)]
(reset! !x (inc x)))
. reset! will re-run if any of its arguments change, so this is a cycle.
The solution for atoms is to use the more typical (swap! !x inc).
The same solution in the database world would be transaction functions I guess.#2023-08-0814:46xificurCif you really wish for the cycle to exist today you can break the cycle with e/snapshot. The tightest fix would be (inc (e/snapshot some-value))#2023-08-0814:55joshchoOh, interesting. That's perfect for my use case. Why doesn't e/snapshot return some-value at the point of when App was mounted?#2023-08-0814:56xificurCI don't understand the question, can you elaborate?#2023-08-0814:59joshchoYeah, I would expect (e/snapshot some-value) to always return 10, since it was frozen at that initial value.#2023-08-0815:01xificurCI see. ui/button mounts the e/fn when you click it and unmounts it when the e/fn returns a non-Pending value. That means the code inside it is mounted anew on each click, thereby running a new snapshot. Does that make sense?#2023-08-0815:02joshchoThat makes a lot of sense. Thank you! Saved me hours of headache.#2023-08-0816:14braai engineerooh didn't know about e/snapshot#2023-08-0816:19xificurCwe don't promote it too much since we're hoping it won't be needed in the next iteration. But for now it's a reasonable weapon in one's arsenal#2023-08-0817:04braai engineerAnyone have an Electric + Reitit routing example lying around?#2023-08-0907:19nakkayaIf you search my messages I should posted couple of times, let me know if you can't find it.#2023-08-0907:33braai engineerThanks! What do I search for? Can’t seem to find in channel for term “reitit”#2023-08-0907:35nakkayahttps://gist.github.com/nakkaya/7b65ddc161aad9beb7207c8b520c47f3#2023-08-0907:35braai engineerFound this cljs mention but looking for server-side: https://clojurians.slack.com/archives/C7Q9GSHFV/p1687788322928669?thread_ts=1687708239.825519&channel=C7Q9GSHFV&message_ts=1687788322.928669#2023-08-0907:35braai engineerThanks, @U5H4U2HEH !#2023-08-0907:36nakkayaI use Reitit for both client side and server side routing, had no issues so far.#2023-08-0907:55braai engineer@U5H4U2HEH does "FIXME: Authenticate WS Connection" mean that WS connections are allowed even without auth? I guess fine as long as you check for session in Electric?#2023-08-0908:22nakkaya> I guess fine as long as you check for session in Electric?
That is my assumption also, a while back I also confirmed this with @U09K620SG. That said I would feel better if I can figure out how to check for authentication when ws connection opens.#2023-08-0910:25Dustin Getzauthentication is intended to be done in userland so you can render a login page with electric#2023-08-0910:26Dustin Getzsee demo-chat-extended#2023-08-0910:47nakkayaThats pretty much how I handle it but ws connection still can be initiated when user is not logged in so I check the session in electric to make sure user is authorized. This is not related to electric but if anyone knows how to configure jetty so it only allows authorized ws connections I'll be great full.#2023-08-1010:13braai engineer@U5H4U2HEH wrap-content-type middleware seems to be overriding Content-Type to application/octet-stream which prompts browser to initiate download prompts. Surprising because wrap-content-type should only override if Content-Type header is not present.
Do you have a more complete sample, please?#2023-08-2209:29braai engineerFor future Reitit searches, most up-to-date is here via @U050CJFRU: https://clojurians.slack.com/archives/C7Q9GSHFV/p1687788322928669?thread_ts=1687708239.825519&cid=C7Q9GSHFV#2023-08-0819:23joshchoHow do you scroll to bottom when you mount? Since dom/on "mount" doesn't exist, I am trying the following:
(dom/div
(dom/props {:class (tw "overflow-auto max-h-96")})
(set! (.-scrollTop dom/node)
(.-scrollHeight dom/node))
;; rest...
)
It doesn't quite work, and not sure where to proceed from here.#2023-08-0820:02Dustin Getzsave a reference to dom/node and then run your side effect at the deepest part of the view, you can use a macro to make the syntax better#2023-08-0820:03Dustin Getzdeepest = causally last child leaf of the view#2023-08-0820:03Dustin Getzoff the top of my head#2023-08-0820:27henrikStick an empty div at the bottom and (.scrollIntoView dom/node (->js {:block "nearest"}) in it.#2023-08-0820:27henrikOr cljs->js, I use cljs-bean.#2023-08-0908:43henrikHere’s another Thing:
(e/defn DelayShow
"Observes the value of an atom, presumed to contain a boolean.
Returns a tuple of booleans: [mount show].
`mount` is used to mount/unmount subgraphs.
`show` is used to trigger CSS transitions, such as changes in opacity."
[atom ms]
(let [value-flow (m/relieve {} (m/watch atom))]
(new (m/latest (fn [^boolean mount ^boolean show]
[(or mount show)
(and mount show)])
value-flow
(m/reductions {} false
(m/ap (let [v (m/?< value-flow)
delay (if (true? v) 1 ms)]
(m/? (m/sleep delay v)))))))))
I use this as a direct replacement for things like https://headlessui.com/react/transition in React land.
I’ve stuck it in an e/defn just to be able to instantiate it with .. I’m still not settled on the patterns I use for various things, but the latest thought is to have all reactive things “look” the same, so as to signal their nature. Just remove new, change e/defn and use new on the call site if this unsettles you.
When atom turns true, there is just enough delay for the subgraph to mount (1ms), after which show flips from false to true. This ensures that the subgraph is instantiated with the correct values, before flipping them and triggering the transitions. On unmount, show flips to false before mount. mount waits for the time needed to run the exit transitions (provided you’ve given it enough of a delay to do so: ensure that this value is the same amount as your CSS transition duration) before it flips to false, removing the subgraph.
Edit: @xifi supplied an alternative version,
(defmacro after
"2-arg: throw Pending for `ms`, then return `form`
3-arg: return `init` for `ms`, then return `form`"
([ms form]
`(case (new (e/task->cp (m/sleep ~ms)))
~form))
([ms init form]
`(case (new (e/task->cp (m/sleep ~ms ::done) nil))
::done ~form ~init)))
(e/defn UseDelayShow
"Observes the value of an atom, presumed to contain a boolean at all times.
Returns a tuple of booleans: [mount show].
`mount` is used to mount/unmount subgraphs.
`show` is used to trigger CSS transitions, such as changes in opacity."
[open ms]
;; Read "after `ms`, turn from `(not open)` to `open`"
(let [ready (after (if open 1 ms) (not open) open)]
[(or open ready)
(and open ready)]))
after can be found in the latest version of Electric, but contains a bug, so a fixed version is inlined above. after is also experimental, so use with caution (and report any strangeness you observe). Note that this version takes a boolean, so supply the return value of (e/watch …) rather than an atom.
So for example (using Tailwind):
(e/defn Popup
[]
(e/client
(let [open-atom (atom false)
[mount show] (DelayShow. open-atom 300)]
(dom/button
(dom/on! "click" #(reset! open-atom true))
(dom/text "Open"))
(when mount
(dom/div (dom/props {:class ["z-60" "fixed" "inset-0" "bg-gray-1000" "bg-opacity-25"
"flex" "justify-center" "items-start" "p-4" "sm:p-6" "md:p-20"
"transition-opacity" "transform-gpu" "duration-300"
(if show "opacity-100" "opacity-0")]})
(dom/on! "click" #(reset! open-atom false))
(dom/div (dom/props {:class ["min-w-[20rem]" "sm:min-w-[32rem]" "max-w-screen-md"
"drop-shadow-xl" "bg-white" "rounded-lg" "overflow-hidden"
"border" "border-gray-200"
"transform-gpu" "duration-300"
(if show "scale-100" "scale-95")]})
(dom/on! "click" (fn [^js e] (.stopPropagation e)))
(dom/div (dom/props {:class ["p-4"]})
(dom/text "Hi!"))))))))
Result is in the video.#2023-08-0909:32joshchoThis is really neat! I don't have much experience with React transitions; how much cleaner is the code/abstractions compared to it?#2023-08-0909:38henrikThank you!
I haven’t actually read the code behind the Headless implementation, but I’d have to imagine that it’s bigger at least.
But it’s also a different abstraction. They provide components which you wrap around your code.
As you might have seen in other threads, I’m a fan of extracting behaviours into small packages with specific purposes, and then composing them with presentation code at the edges.#2023-08-0913:47telekidGiven https://clojurians.slack.com/archives/C7Q9GSHFV/p1653828008842959?thread_ts=1652373556.897869&cid=C7Q9GSHFV, I'm generally following a pattern where atoms are turned into flows immediately and functions take flows as arguments rather than atoms.#2023-08-0913:48telekidCurious to know if I'm thinking about this in the right way.#2023-08-0914:15telekid(This is really useful btw)#2023-08-0914:28henrikOriginally, the function took a flow, that’s why there’s a vestigial and probably useless relieve there (as I couldn’t guarantee that we wouldn’t be absolutely bombarded by bools, given that it’s arbitrary).
I switched to m/watch, because in practice I’m not using this with anything but an atom at the moment, and I figured that it’s more approachable. If it takes a flow, the user has to figure out how to convert the bool to a flow (sure, just wrap e/fn), or use m/watch. That felt like over-generalization given its purpose. Also, you’re meant to use the [mount show] return values, rather than the atom value directly. They kind of supersede the atom value.
But I don’t know, any notes @U09K620SG? Particularly, the fact that I’m taking from the m/watch flow in two places?#2023-08-0916:47Dustin Getzyeah the m/watch as written will subscribe twice, which you don't want. It will introduce inconsistency – you'll get two reactive propagations instead of one and the first propogation results in your app seeing an undesired intermedate state where one watch has already fired but the second one is about to#2023-08-0916:49Dustin GetzWhy? missionary flows like (def >x (m/watch ...)) are referentially transparent, which means they will run the flow and any effects (establish the watch) each time they are consumed. So here the watch will be created twice as it is consumed twice. The structure here to imagine is a "process supervision tree" , you have embedded the flow twice in the tree, and the flow is referentially transparent (a recipe describing how to subscribe to an atom), and the recipe will run twice#2023-08-0916:50Dustin GetzTo make the result to be shared you need to wrap it like (m/signal! (m/watch ...)) which will allocate a memo buffer ("DAG node") and share the value if consumed from N places (thus not repeating upstream effects, like subscribing to the atom)#2023-08-0916:50Dustin GetzGuidance: prefer e/watch to m/watch for two reasons:
(1) e/watch will establish the watch once and share the result
(2) it is forward-compatible if we change the electric/missionary boundary#2023-08-0916:52Dustin Getze/watch has different behavior because Electric let forms (like (let [x (e/watch !x)] ...) compile to m/signal!, i.e. x is an m/signal! - in other words, electric let allocates memory in the reactive heap, let is the memoization point#2023-08-0916:53Dustin GetzIn electric if you write (let [x (m/watch !x)] you memoize the recipe for the watch, which is not what you want (the ref is constant/invariable anyway)#2023-08-0916:54Dustin Getzif you write (let [x (new (m/watch !x))] you memoize the values coming out of the watch, because new runs the flow and joins it. and (new (m/watch !x)) is what e/watch macroexpands to if you check the source.#2023-08-0917:02Dustin GetzRegarding backpressure, (m/relieve (m/watch !x)) is not really the right place, you'd want to colocate the relieve with the actual user event (foreign platform callback) which is dom/on! which calls e/listen> which already bundles the m/relieve#2023-08-0917:03Dustin Getz#?(:cljs (cc/defn dom-listener [node typ f opts]
(.addEventListener node typ f (clj->js opts))
#(.removeEventListener node typ f)))
#?(:cljs (cc/defn listen> ; we intend to replace this in UI5 workstream
([node event-type] (listen> node event-type identity {}))
([node event-type keep-fn!] (listen> node event-type keep-fn! {}))
([node event-type keep-fn! opts]
(m/relieve {}
(m/observe (cc/fn [!]
(dom-listener node event-type #(when-some [v (keep-fn! %)]
(! v)) opts)))))))
#2023-08-0917:06Dustin GetzActually I made a mistake – in your snippet, you're not using the return value of dom/on! which is the thing that is relieved#2023-08-0917:09Dustin Getzi might write refactor (dom/on! "click" #(reset! open-atom true)) to (when-some [e (dom/on! "click" identity)] (reset! open-atom true)) which gets you the bundled relieve and moves closer to the functional style#2023-08-0917:14Dustin Getzthe when-some is because the initial value of the event is nil. Probably that initial nil inside dom/on! is better written as (throw (Pending.)) which will defend against an accidental NPE. But now you start to see how Pending and "Loading" are not quite the same thing, we have more work to do here#2023-08-0917:56henrikWow, thanks for the detailed feedback, and insights into Electric!
So, two potential versions:
;; Alt 1, use m/signal!
(defn delay-show-1
[atom ms]
(let [value-flow (m/signal! (m/watch atom))]
(m/latest (fn [^boolean mount ^boolean show]
[(or mount show)
(and mount show)])
value-flow
(m/reductions x/discard false
(m/ap (let [v (m/?< value-flow)
delay (if (true? v) 1 ms)]
(m/? (m/sleep delay v))))))))
(e/defn DelayShow1
[atom ms]
(new (delay-show-1 atom ms)))
;; Alt 2, bridge e/watch
(defn delay-show-2
[value-flow ms]
(m/latest (fn [^boolean mount ^boolean show]
[(or mount show)
(and mount show)])
value-flow
(m/reductions x/discard false
(m/ap (let [v (m/?< value-flow)
delay (if (true? v) 1 ms)]
(m/? (m/sleep delay v)))))))
(e/defn DelayShow2
[atom ms]
(let [value (e/watch atom)]
(new (delay-show-2 (e/fn [] value) ms))))
For alt 2, does e/fn still stand as the best way to bridge the e/watch return value to a flow for input to the helper function? Other than that, I’ll take your direction that leaning on e/watch is the more future proof way of going.
Regarding relieve and backpressure: I get that, eliminate the problem at source. When it comes to reusable, library style functions, I tend to build for an imaginary person who is in a rush, doesn’t read the source code, will copy example code by rote, etc. Hence a relieve here in order to be as structurally sound as possible in and of itself.
Regarding dom/on!: Yes, I’d prefer the functional approach, but wrapping every single dom/on! in when-some is too much boilerplate for my liking. Peter guided me toward this approach the other day:
(defn on*
[node type f]
#?(:cljs (m/reductions x/discard (Failure. (Pending.))
(e/listen> node type f))))
(defmacro on!
[type f]
`(new (on* dom/node ~type ~f)))
Nevertheless, the solution above not standard Electric, so I left the dom/on! functions in the example side-effectful instead.#2023-08-0917:59telekid> To make the result to be shared you need to wrap it like (m/signal! (m/watch ...)) which will allocate a memo buffer
and
> f you write (let [x (new (m/watch !x))] you memoize the values coming out of the watch
@U09K620SG I'm confused by the difference between m/signal! and new here#2023-08-0918:03henrikLooking at the docstring of m/signal!, it sets up a subscription/broadcasting style frame around a flow. So I understand it as instantiating the flow it wraps once, and then establishing a subscription style mechanic for each instantiation of itself when called with new.
The second one, used in an e/defn, will do roughly the same thing, but Electric handles the m/signal! part under the covers.#2023-08-0918:10henrik(Check the source of e/watch, it’s essentially (new (m/watch)) with some assertions)#2023-08-0920:03Dustin Getz@U06B8J0AJ DelayShow2 (lifting the flow with e/fn) is better, the only valid reason IMO to pass around a reference is to write to it#2023-08-0920:07Dustin GetzI think i was able to code golf it down to this, I have not tested. The key is that the transition controller flow can just be terminated and rebuilt if the atom changes, right?
(defn transition-delay [open ms] (m/sleep (if (true? open) 1 ms) open))
(defn transition-controller [open ms]
(m/ap
(m/amb [open false]
(let [show (m/? (transition-delay open ms))]
[(or open show)
(and open show)]))))
(e/defn Popup
[]
(e/client
(let [!open (atom false) open (e/watch !open)
[open show] (new (transition-controller open 300))]#2023-08-0920:07Dustin Getzhere i used m/amb to provide the initial value rather than m/reductions#2023-08-0920:10Dustin Getz~in fact the m/ap is not needed at all here as the soln does not use m/?<~, maybe i have made a mistake, unfortunately no time to continue
*the m/ap is needed around m/amb which returns multiple values#2023-08-0920:28Dustin Getzthe 1ms delay is bothering me too, can we move the transition-controler "to the bottom" ? Doing so reveals a fun issue which is you need to "loop" the flow it returns back to the top in order to get it back into lexical scope where it is used#2023-08-0920:34Dustin Getzyou can use an atom to loop the value back up for now, the last time the we considered this (how to permit mutually recursive let bindings) it was a bit of a hot topic, we'll need to hash it out at some point#2023-08-0920:35Dustin Getzhaskell has an extension RecursiveDo which enables mutually recursive bindings in monad comprehensions for certain monads which is essentially the structure in play here#2023-08-1006:41henrikAs long as it behaves correctly, it doesn’t matter if it’s destroyed and recreated during it’s runtime, though it sounds a bit bizarre to me. But that might be due to my partial understanding of Missionary. A pipe that takes a value and emits another value, remains in place, and isn’t reconstructed until it’s not needed anymore sounds more logical to me.
So alt 2 is fine, I think, there’s no particular need to yak shave it down (though it might be an interesting exercise). It’s not a performance-intensive piece of code.
As for the 1ms thing, I don’t think using the function should be a more elaborate ritual than it is. Partially because of usability, partially because I’d probably consider it a regression to have it become any more complected with domain specific code than it is. Currently, it hardly knows anything about your code, and that’s a good thing.
If there were a general way of checking that the subgraphs dependant on the bool had been executed fully (including any DOM elements mounted with their CSS classes applied and done), rather than contortions specific to this function, that would be an interesting mechanism to hook into. But then it would be general—the same could be used to do any number of other things, such as showing and hiding a whole-app spinner for example, and not particular to what’s going on here.#2023-08-1007:03henrikAnother way to look at the 1ms thing is: I need to know that “false” case graph has been processed fully before it processes “true” case graph. If there’s a general mechanism that allows me to queue up “false”, and “true”, and know that the first will be processed fully (including DOM node mounting) before “true” is processed, that would be fine as well, and would require no knowledge internal to the function of how the bool is used externally.
And I think all of it should be considered from the point of view of: what level of mastery should it require to extend the behaviour of Electric with something like this? The more boilerplate there is, and the more I need to “spread” the internals of this functions across a specific code site where it’s used, the less portable it is.
I understand that you want to deliver UI components and functions that are correctly written to the level of your own mastery of the subject. But if you want to build a thriving community that extends it beyond what you are capable of on your own in terms of reach (like React or Vue has, for all their shortcomings), it must be possible to write good-enough-gets-the-job-done pieces of code AND have them be sharable and portable with codebase-specific convolutions kept as low as possible. Otherwise it’s probably not going to travel far from where it was written, and there’ll be less extension of Electric at a community level.#2023-08-1007:16henrikSo I guess a summary of considerations are:
1. What are the patterns I should reach for when extending the behaviour of Electric? How do we make them obvious? From the discussion we’ve had so far around this bit of code, it’s clear that there are at least some subtleties that are fairly opaque.
2. Is it possible to set patterns so as to draw cleaner lines between behaviour extensions? The more composable and less complected a particular extension needs to be with surrounding code, the more portability and impact it will have.
I think level of mastery required can be considered orthogonally from this, because ultimately it’s OK if Electric requires a higher level of mastery than React etc., it’ll just occupy a different niche.#2023-08-1008:00xificurCIIUC the Thing derives a new value based on a reactive boolean. If the bool is true the derived value turns true after 1ms, if it's false after user-provided ms.#2023-08-1008:09xificurCthe question "how can I do Y when all of X is completed" (in this case it's "how can I do something after all of the DOM has mounted") comes up a lot. We understand the use case and that achieving it seems more complicated than it needs to be.
The problem is "after all of the DOM has mounted" is very opaque in a reactive language. The DOM is live and reacts to all sorts of events. I think if you were to give a precise definition which point in time do you consider "the one" you'd find it's harder than it looks and might miss certain edge cases.
If there is a specific point where you know the mounting is finished you can (reset! !done true) at that point.#2023-08-1008:59henrikYeah, I think you’ve captured it. It’s complicated, and it’s not easy to move from 1ms to predictably reactive without considerations custom to the particular usage site.#2023-08-1009:02henrikThat’s why I’m talking about patterns. IMO there’s either a general pattern to lean on, or 1ms will have to be OK until it’s not.#2023-08-1009:14henrikYour suggestion isn’t behaving quite as expected:
;; Opening…
{:open? true :ready? nil
:res [true nil]}
{:open? true :ready? true
:res [true true]}
;; Closing…
{:open? false :ready? nil
:res [nil false]}
{:open? false :ready? false
:res [false false]}#2023-08-1009:15xificurCif you can signal "ok now" you can create a mailbox (`m/mbx` ) and send a message. Then in DelayShow instead of 1ms you can poll the mailbox#2023-08-1009:15henrikOpening is OK (other than coercing to bools), closing is abrupt.#2023-08-1009:17henrikmbx feels side-effecty to me. I could do all this with an atom and js/setTimeout as well, but I don’t know if any of it is worth it.#2023-08-1009:18xificurCyou're right, the macro has a bug, ~pending and nil need to switch places#2023-08-1009:18xificurCif that doesn't work I'll debug it locally#2023-08-1009:19henrikYep, that did it#2023-08-1009:20henrikNow it also emits the bools as expected instead of nil.#2023-08-1009:20henrikNice, I like this solution. after feels like a general and good function.#2023-08-1009:21xificurCyou might be able to use a reactive value instead of mbx as well#2023-08-1009:21henrikWhat’s the purpose of the pending parameter to after?#2023-08-1009:24henrik(not open) is the pending parameter. I’m not following along.#2023-08-1009:24xificurCmaybe init is a better name. The 2-arg version throws Pending and after ms resolves to form. The 3-arg version returns init for ms and then resolves to form#2023-08-1009:24henrikAh, gotcha#2023-08-1009:27xificurChttps://github.com/hyperfiddle/electric/commit/6104ea67a1f066c9f51dc334ab011707357cf717#2023-08-1009:28xificurCnote that right now it's experimental and unused in our codebase#2023-08-1009:29henrikOK, well it’s going to be used in my codebase in the meantime 🙂#2023-08-1011:25Dustin Getz> If there were a general way of checking that the subgraphs dependant on the bool had been executed fully (including any DOM elements mounted with their CSS classes applied and done)
the DAG defines exactly a causal ordering of effects, we write (do (println 1) (println2)) not (do (future (Thread/sleep 1) (println 2)) (println 1))#2023-08-1011:27Dustin Getzi think it matters because the 1ms magic constant is vulnerable to fluctuations in the user environment such as: js gc pause, OS cpu scheduler, low laptop battery, resource contention with other tabs, etc#2023-08-1011:42henrikRight, but there’s no causality between the bools. I could, I guess, emit functions which return the bool rather than the bool directly, and which report back their lifecycle..
I could wait with the bool flipping for each individual site based on their individual reporting, ensuring at least that the underlying DOM node has mounted, but I probably couldn’t flip them all synchronously, as I don’t know up-front how many sites you’re going to apply it to.#2023-08-0910:49braai engineerWith RCF, what can I do on the right side of := ? E.g. I want to say (do-thing) := [int? int?] or enforce some kind of spec.
(s/valid? (do-thing)) := true feels kind of clunky and I can't see what the output actually is, only that it is (not true).#2023-08-0911:25Dustin Getzwe have many ideas but can't invest in them until after Electric is finished#2023-08-0911:25Dustin GetzYou can register custom test predicates – see source#2023-08-0911:25Dustin Getzyou can also write macros that contain tests inside of it as well as :=#2023-08-0911:26Dustin Getzfor example https://github.com/hyperfiddle/rcf/issues/50#2023-08-1010:32joshchoI took a stab at optimistic updates using atoms, would appreciate any feedback. The following is non-optimistic code for resetting a value with a button, assuming Datomic-like db.
(let [id 42
value (e/server
(e/watch
(:some/value (d/entity db id))))]
(ui/button
(e/fn []
(e/server
(d/transact! !conn [{:db/id id
:some/value
(if (= value "A")
"B"
"A")}])))
(dom/text value)))
This creates a button that resets some part of the database. This is not optimistic; if there's any network delay, we would see changed value after d/transact! has completed. If we wanted to make this optimistic, we can fiddle around with ui4/control, but it's messy (maybe different with ui5?).
I implemented a macro relay-atom which adds optimism without much clutter. Using relay-atom to make the above optimistic:
(let [!value (relay-atom
(:some/value (d/entity db 42)) ; server query
(fn [v] ; server effect
(d/transact! !conn [{:db/id 42
:some/value v}])))
value (e/watch !value)]
(ui/button
(e/fn []
(if (= value "A")
(reset! !value "B") ; triggers server change
(reset! !value "A") ; triggers server change
))
(dom/text value)))
With relay-atom, any change to !value (e.g. reset!, swap!) is propagated to the server, and any change to the server (by different client) is propagated to relay-atom. The implementation detects and stops cycles in updates.
Here's the https://gist.github.com/joshcho/9549e8b35acaec2815ae6436adb02a79.#2023-08-1010:34joshchoI have also added a convenience macro e-relay-atom for accessing attributes of entities in Datomic-like db, so the above code can be simplified even further:
(let [!value (e-relay-atom 42 :some/value)
value (e/watch !value)]
(ui/button
(e/fn []
(if (= value "A")
(reset! !value "B")
(reset! !value "A")))
(dom/text value)))
which imho abstracts away the optimism part completely.#2023-08-1010:37Dustin Getzui4 does not lose state in the presence of multiplayer concurrent modifications #2023-08-1010:37joshchoRight, I use ui4 for things like input already, but there are special cases where exposing the atom is useful for my use case.#2023-08-1010:38Dustin Getzas long as it doesn’t have bugs i’m good with it#2023-08-1010:39joshchoOut of curiosity, how will optimistic updates in ui5 look like?#2023-08-1010:40Dustin Getzoh i see now you’re still calling into the ui4 infrastructure just a different user facing api#2023-08-1010:40joshchoyeah, using e/for-event#2023-08-1010:42joshchoI think this can be useful API for when users want a little more than what's provided by ui/input, etc. Basically ui/control with exposed atom#2023-08-1013:57Dustin GetzUI5 is a callback free pattern, a text input control is a signal, it returns its state as a reactive value. The state is propagated upwards to the entrypoint where there is a single transact. Controls, fields and forms compose their states, turning the values into transactions on the way up. There are some fancy new electric features and improvements coming that make the pattern work (upon which the UI5 workstream is currently blocked). The form/field/control composition can further compose into collections/tables, including optimistic maintenance of a CollectionView for a "create new entity" operation like in TodoMVC#2023-08-1014:03Dustin GetzThe reactive value that the entrypoint sees – the "form" – can be thought of also as a "monitor" in that it is basically a view of pending operations that the server hasn't acknowledged yet. A series of rapid edits can either be raced into the database as the user types, OR any layer in the composition can intercept/buffer the "form" for transactional behavior – say you want to control when a value is committed (giving you an opportunity to hit esc and cancel the change or hit tab/enter to submit, like TodoMVC and google sheets). Or say you want to have transactional form edits in a modal with form-level validation logic, the popover can intercept/buffer the form state until the user indicates "save now" at which point the edit is allowed up#2023-08-1014:04Dustin GetzThere is a feedback mechanism so that controls understand when their edit has been committed thus clearing their internal state (the pending/optimistic value that they retain until commit is acknowledged)#2023-08-1014:07Dustin Getzincluding handling of edge cases like, a control receives a server ack for the right value but as transacted by someone else, it was not their own edit that produced the expected value, and therefore they are still pending because their transaction may have had other side effects that haven't happened yet#2023-08-1014:38joshchoVery cool! Looking forward to it.#2023-08-1018:50VincentMy development setup with Electric is pretty cool.#2023-08-1018:50VincentI have my live box and for most things can update the code on the live box. [live tweaks]
But, for things that take more iterative development (most if not all the application fits this description)
It is good to be able to swiftly copy the whole codebase to another machine and work locally.
So, I have it such that if I brush away the entire data folder (xtdb database) and download it from my live box, I effectively have an up-to-date working copy of the data.
And for the code/files, just gotta pick which files I want to overwrite the local copy with the live edition.#2023-08-1018:51VincentEvery local machine has its own "git" and when i am iteratively developing locally I save to the local git branches. When there's a big update I just bludgeon files, sometimes there is a merge-conflict dance on the local machines, but it's manageable#2023-08-1018:52VincentUsually I copy new code in component-by-component, while updating local branches from the live edition via scorched-earth approach#2023-08-1018:53VincentAnyway, coding in electric is a delight and I hope to eventually be showing more people how fun and straightforward it can be#2023-08-1019:46weihello, noob here, greatly enjoying Electric so far. is there a way to directly inject html as a string?#2023-08-1019:48Dustin Getz(set! (.-innerHtml dom/node) “”)#2023-08-1019:55weithanks for the reply! where does dom/node come from?
e.g. I'm doing this and getting an "illegal invocation" error
(dom/div
(set! (.-innerHtml dom/node)
"<svg version=\"1.1\"\n width=\"300\" height=\"200\"\n xmlns=\"\">\n\n <rect width=\"100%\" height=\"100%\" fill=\"red\" />\n\n <circle cx=\"150\" cy=\"100\" r=\"80\" fill=\"green\" />\n\n <text x=\"150\" y=\"125\" font-size=\"60\" text-anchor=\"middle\" fill=\"white\">SVG</text>\n\n</svg>"))
#2023-08-1019:56wei(the html string comes from some other source so i can't use the svg macros)#2023-08-1019:56Dustin Getzprobably i didn't write exactly the right property#2023-08-1019:57weiah! innerHTML#2023-08-1019:57weiworks! i see you answered something similar a couple times before, thanks for taking the time to help me out#2023-08-1020:03Reut SharabaniJust wanna say that it's really inspiring to see Dustin helping so many people directly every day.#2023-08-1020:05JOh yeah! And all other members of the team! Congrats guys!#2023-08-1107:52henrikI’d like to take a value from a mailbox as often as there is one, and when there is not a value available, I’d like to Pend.
This is wrong, what’s the correct way?
(m/ap (loop []
(m/amb
(Failure. (Pending.))
(m/? mbx)
(recur))))#2023-08-1107:58xificurCthis takes a value from an mbx and throws pending while waiting
(new (e/task->cp (m/? mbx)))#2023-08-1107:58henrikI started there, but It dies after taking the first value.#2023-08-1107:59xificurCthere's no loop there, yes. Can't you loop from the outside?#2023-08-1108:00henrikIf I wrap it in a loop, how do I yield a value from that loop?#2023-08-1108:02henrikI.e., I’d need the return to be a stream of <pending value pending value pending>. Or I think I’d end up effectively looping the body of an e/defn#2023-08-1108:05leonoelYou can "poll" a mailbox with race :
(m/? (m/race mbx (m/sp :timeout)))#2023-08-1108:11henrikThanks @U053XQP4S, but polling seems odd in this scenario. Is there no way to return two successive values on an ap per step of a loop?
I imagine a pipe where a value is immediately followed by Pending, until a new pair is returned.#2023-08-1108:11henrikHang on, some background:#2023-08-1108:13henrikThe usecase is that I’m returning a user selection in a reactive function that provides menu functionally. So, rather than side-effecting inside an on-select function or similar that you pass in, I’d like to return selections as a stream of values. And I might to react to that selection by, for example, writing to the DB.
However, let’s say that I’m also sending along the current mouse coordinates to the transaction function, or something else that updates more often than I want to transact.
My thought is that if I make the selection input pend, except for at the precise moment that a new selection is returned, the mouse coordinates updating will not trigger re-transactions.#2023-08-1108:15henrikSince there are multiple ways to select in a menu, I put the selected value in a mailbox.#2023-08-1108:18henrikI mocked it up by storing the selection in an atom and then essentially
(reset! select-atom selected-item)
(reset! select-atom (Failure. (Pending.)))
With an (e/watch …) it, works, but reads icky. There must be a better way to produce a stream of successive values.#2023-08-1109:22xificurCelectric runs in continuous time, relying on "precise moment" is unreliable, i.e. if a discrete flow returns a value and Pending right afterwards you might not observe the value at all#2023-08-1109:25xificurCreturning 2 values in discrete time could be accomplished in an m/ap by (m/amb (m/? mbx) (Failure. (Pending.)))#2023-08-1109:26xificurCif it's the mouse coordinates that updates too fast I'd think about throttling that stream of values instead#2023-08-1110:33Dustin Getzto do discrete things you can rely on edge detection (works today in electric because the entrypoint samples eagerly as fast as possible with no throttling by requestAnimationFrame). Or you can return a missionary discrete flow up the stack (bypassing electric altogether).#2023-08-1110:34Dustin Getzi don’t understand the issue with mouse signal interaction, why pass mouse signal as param to transact#2023-08-1110:35Dustin Getzyou can use e/snapshot or n/sample to sample the mouse coordinates with a discrete flow#2023-08-1111:04henrikMouse coords were just an example of something that changes more often than item selection. I chose this as an example to contrast it to the entirely user intention driven event that is clicking something in a list with, with regards to transact.
Since I’m modelling the equivalent semantics as an event side-effect, I’d like to handle this at the source rather than leaving it to the consumption side. I’m 100% sure that omitting behaviour, and leaving it up to everyone to handle it on a case by case basis is going to lead to subtle bugs, unless I deliver the semantics of an event firing.#2023-08-1111:07henrikConsider dom/on, what would it be like if it fired continuously for every source of data you might reference in it, rather than just on the named event?#2023-08-1111:24henrikAnyway, thanks for the input. To make my particular example exhibit the behaviour I’m looking for, the following worked:
(->> (loop []
(m/amb
(Failure. (Pending.))
(m/? mbx)
(recur)))
(m/ap)
(m/stream!))
I still have limited understanding of both Missionary and Electric, so I don’t know if this would count as an idiomatic solution or not.#2023-08-1111:42henrikDoes Electric automatically relieve continuous flows when it encounters them?#2023-08-1112:24leonoelstream! is likely not necessary and will generate memory leaks#2023-08-1112:25leonoel> Does Electric automatically relieve continuous flows when it encounters them?
no#2023-08-1112:30xificurCit's m/latest that's everywhere.
Consider the example
• m/latest is notified from upstream of a new value (a mailbox value)
• m/latest notifies downstream that a new value is ready for transfer
• downstream transfers from m/latest
• m/latest transfers from upstream (your mailbox value)
• during that transfer upstream notifies of a new value (Pending)
• m/latest transfers from upstream (Pending)
• m/latest returns Pending downstream
Downstream never sees the mailbox value#2023-08-1112:34henrikAh, I see. OK, then I’m no closer to understanding how to predictably return both of them.#2023-08-1112:39xificurCcalling @iter will only ever return :pending#2023-08-1112:41xificurC> I’m no closer to understanding how to predictably return both of them
the answer is probably you shouldn't. It doesn't make sense in continuous time and you need a different approach#2023-08-1112:41henrikAlright, I’ll return to side-effects. Thanks anyway!#2023-08-1112:43Dustin Getztake a hard look at https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_compiler.clj to understand the electric to missionary continuous flow transform , this is pseudocode and omits the network instrumentation but is helpful imo#2023-08-1112:45henrikThanks @U09K620SG, will do.
dom/on exhibits the behaviour I’m after. Is it special?#2023-08-1112:47xificurCit takes a callback, mounts it and unmounts it once it returns a non-Pending value. In that regard it's also very side-effecty#2023-08-1112:48henrikSure, but what it does internally doesn’t really matter as long as it presents a functional interface. So for all intents and purposes, it’s correct IMO.#2023-08-1112:48xificurCyou're right that it's trying to solve a similar problem, bridging discrete events into continuous time#2023-08-1112:55xificurC> My thought is that if I make the selection input pend, except for at the precise moment that a new selection is returned, the mouse coordinates updating will not trigger re-transactions
can't you sample the "mouse coordinates" only when the input changes?#2023-08-1112:57henrikYes, but I’m going to put that burden on every usage of the selected item. I’m sure it will lead to confusion and bugs. Consider if you’d put that burden on everyone for dom/on.#2023-08-1112:59henrikSemantically, on-select IS an event. I’m just trying to give it a functional, non-side-effecty interface. Like dom/on. It seems in the spirit of Electric to at least attempt it.#2023-08-1113:02Dustin Getzyou have reached the frontier of our idioms, we are still working it out. the official guidance is to use dom/on if you need to while we get UI5 finished which is a callback free pattern. UI5 is blocked on differential electric, which Leo is actively finishing we hope to be testing it in september#2023-08-1113:04Dustin Getzthe side effect callback vs pure signal has been fiercely debated internally#2023-08-1113:04henrikAh, OK. To be clear about this particular use-case though, I’m not missing any particular UI component you might have in the works in that case. I’m missing the discrete->continuous interface that you might build an arbitrary component on.#2023-08-1113:05xificurCI think we've reached a point where a minimal runnable example would be required to increase communication bandwidth#2023-08-1113:05henrikSure, I’ll see if I can put one together.#2023-08-1113:06Dustin Getzfwiw i have access to Henrik’s repo and spent some time coming up to speed in it but unfortunately am traveling through this weekend #2023-08-1113:09henrikI have a ton of uncommitted code related to this anyway, @U09K620SG. Good to hear that there’s no pressure to clean it up 😅#2023-08-1113:09Dustin Getzthinking about it, i think the parts of UI5 that Henrik needs could be delivered without differential electric, it is just currently entangled with the optimistic update design which does require differential electric. We could escalate it if we decide to do that thing we talked about#2023-08-1113:11henrikOh boy, yeah think it through. It’s not that I’m blocked or can’t use side-effects, it’s just that I’m enamoured with the principle of functionally returning values (providing the equivalent semantics of an event), rather than side-effecting.#2023-08-1113:16Dustin Getzdom/on was a high bang for buck bandaid#2023-08-1113:17henrikCertainly, by turning the initial nil it emits on mount to a Pending, it exhibits exactly the right behaviour for my purposes.
I went a step further and just gave it a single [node type] signature, as everything else can be handled on the return value of it.#2023-08-1114:08henrikHere’s a severely shrunk and mangled version of my menu function that I think demonstrates the core question.
;; Code golf time! The goal is to get "Functional select" and "Side-effect select" to print
;; the same number of times to the console, by _only_ modifying `UseMenu`, and leaving `Menu`
;; as it is.
;;
;; I.e., both ways of reacting to the selected value exhibit the same behaviour.
(defn nil->pending
"Return `Pending` if the value is `nil`. Otherwise, return the value."
[value]
(if (nil? value) (Failure. (Pending.)) value))
(e/defn UseMenu
[items]
(let [!state (atom {:selected nil})
state (e/watch !state)
id-vec (mapv :id items)
selected-idx (.indexOf id-vec (get state :selected))]
{:items (if (= -1 selected-idx)
items
(assoc-in items [selected-idx :selected] true))
:select-item (fn [item]
(let [idx (get item :id)]
(get (swap! !state assoc :selected idx) :selected)))
:OnSelect (let [selected-item (get state :selected)]
(e/fn [] (nil->pending selected-item)))}))
(e/defn Menu
[]
(let [{:keys [items OnSelect select-item]}
(UseMenu. [{:id 0 :title "First"}
{:id 1 :title "Second"}
{:id 2 :title "Third"}])
current-time (int (e/client e/system-time-secs))]
(println
"Functional select:" (OnSelect.)
"clicked at" current-time)
(e/for-by :id [{:keys [title selected] :as item} items]
(dom/div (dom/props {:style {:color (if selected "red" "black")}})
(dom/on! "click" #(println
"Side-effect select:" (select-item item)
"clicked at" current-time))
(dom/text title)))))#2023-08-1114:16Dustin Getzyou’ve substituted the time signal for the mouse coordinate signal? #2023-08-1114:17henrikYeah, like I said, it’s just about having some other input that might change at other times than when you’re just selecting something.#2023-08-1114:18xificurCnitpick, nil->pending turns false to pending#2023-08-1114:19henrikGood point! Not good braining right now.#2023-08-1114:20henrikUpdated with the actual one I’m using. Not sure why I rewrote a poorer version for the example.#2023-08-1114:21Dustin Getz“why are you transacting the mouse coordinates” is relevant to this#2023-08-1114:23henrikIt is?#2023-08-1114:36xificurCthe solution you're looking for goes against electric's behavior. (println (OnSelect.) current-time) depends on 2 reactive values, if any of them changes the println re-runs. The dom/on! example runs once only because the function runs synchronously. If you were to later switch to dom/on and add a server transact you'd also see the println multiple times#2023-08-1114:38xificurCwhat I'm trying to say is this is a fundamental behavior of the system a user has to understand in order to use electric#2023-08-1114:41henrikI’m not sure I follow. So, I changed it to:
(dom/on "click" (e/fn [_]
(println
"Side-effect select:" (select-item item)
"clicked at" current-time)))
And it still prints once per click.#2023-08-1114:42xificurCyou can however sample the current time on a new selection, with (defn time [_] (js/Date.)) and call (let [selected (OnSelect.)] (println "Functional select:" selected "clicked at" (time selected)))#2023-08-1114:43xificurCadd (e/server (new (e/task->cp (m/sleep 5000)))) to the click e/fn#2023-08-1114:44henrikAh, but if I change it to:
(println
"Side-effect select:" (dom/on! "click" #(select-item item))
"clicked at" current-time)
We have the same behaviour.#2023-08-1114:49henrikThen it makes sense. And in order to provide the same behaviour as dom/on, it would take a function. And the behaviour should be that the body of the function given to OnSelect happens once per selection, while the return value of OnSelect is continuous.#2023-08-1115:00henrikAlright, thank you! OK, I will go back and think a little bit about wth makes sense as the interface for selection.#2023-08-1115:04henrikSo if I do need to do something server-side inside an on, I should make damn sure that things are snapshotted#2023-08-1115:07xificurCthere's cases where you want reactivity while dom/on is running (e.g. spinners, progress bars, timers)#2023-08-1115:17henrikThat makes sense#2023-08-1116:46Dustin Getzthe dom/on “callback” is really a piece of DAG and there is machinery inside dom/on that carefully mounts the piece of dag and then unmounts it when the first non-pending value is seen. It is unnatural, i will emphasize, this is not a natural pattern, we are artificially managing the lifecycle of that piece of DAG such that it is alive and mounted for the duration of the network latency and then force unmounted. it is complex and unnatural and needs fixing. historically this pattern was an early experiment in 2021 which we very much want to kill, however there are good reasons- one of which is that it matches the way database transactions work and models the latency states correctly - databases are NOT pure functional and this is the root cause of the complexity here and why we guide you to use the dom/on pattern for now despite it being unnatural.#2023-08-1116:48Dustin Getzreally the only good thing about the pattern is that it works, there are no latency/concurrency bugs, it’s behavior is fully determined, and it works just fine with database transactions without further fuss (once you understand how to combine it with e/snapshot if needed)#2023-08-1116:49Dustin Getzthe pure functional pattern in UI5 has functionality to deal with the impedance mismatch between pure function and database transaction #2023-08-1407:42henriksnapshot looks like it kind of turns a reactive value into a constant. For grabbing the latest value of something. iff some other value changes (like grabbing the latest client timestamp when click event changes), it doesn’t seem like the solution. Or do I misunderstand snapshot?
I.e., the equivalent of something like this in React:
(let [current-time (get-current-time)]
(use-effect [event]
(transact event current-time)))#2023-08-1407:42henrikHow would I model this?#2023-08-1408:04xificurC(foo bar) runs when bar changes (or foo of course). You can therefore re-run a piece of clj(s) code by making it depend on the reactive value
(defn get-current-time [_event] (js/Date.))
;; or inline
((fn [_event] (js/Date.)) my-event)
#2023-08-1408:12henrikRight. The current-time in this case is just a placeholder for a reactive value. Let’s say it’s something else, like a selection in a dropdown, which would be a reactive value.
I want to grab the selection at the moment in time of the click, but I don’t want to re-run the same code at the time when the selection in the dropdown changes.#2023-08-1408:15henrikWhat I can think of right now is to,
1. Make an atom, server-side (or one for each reactive value).
2. reset! the reactive value into those atoms reactively.
3. Send in the atoms to the non-reactive transact function.
4. Non-reactive deref inside transact.
Possibly eliding nil-state of the atom values or something like that in order to skip transact running on mount.#2023-08-1408:15henrikWith only event being passed to transact directly, instead of via an atom.#2023-08-1408:17xificurCare you explaining a single assignment variable here? "undefined" initially, set once?#2023-08-1408:19xificurCif yes that's m/dfv
(let [selection (m/dfv)]
...somewhere (selection (-> event .-target .-value))
(transact (new (e/task->cp selection)))
#2023-08-1408:20henrikSorry. The premise is, I have two reactive values, and a piece of code I want to run when only one of the changes. The other one, I don’t care about, except what the current value is at the moment in time when the first one changes.
So, inputs: event, dropdown-selection, both reactive.
When event changes, run (transact event dropdown-selection). When dropdown-selection changes, do nothing.#2023-08-1408:30xificurCoptions that come to mind
• cc/defn transact memoizes the last event and transacts only when it changes
• dropdown-selection is obtained through non-reactive means (e.g. (cc/defn get-dropdown-value [_event] (.-value (dom/by-id "my-dropdown"))) )
• you pass in a custom object whose equality is defined only via event
• (as you pointed out) the dropdown value is stored in an atom and you deref it#2023-08-1409:03henrikRight, so generalizing from dropdown-selection, any such value must be extracted via the DOM or some other intermediary, mutable state. I can’t go from a reactive value to a moment-in-time value from within Electric itself, as use-effect would provide.
I.e., the solution is the same as with the atom suggestion I had above, with the only difference that you put the value somewhere in the DOM, and I put it in an atom.#2023-08-1409:09xificurCI think a use-effect macro would be possible in electric. It is against electric's nature though. I wouldn't try too hard to bring all the react concepts into electric, e.g. use-effect is strictly less powerful than what electric provides OOTB.#2023-08-1409:34henrikAbsolutely, and I’m for sure not advocating it. I’m just trying to understand how to set conditions on the reactiveness, as above, since I know the need will arise. So it helps to know that intermediary mutable state is necessary to “escape” from the reactiveness of a reactive value.#2023-08-1109:41braai engineerBest way to get current GIT_REV as $VERSION into Dokku because HYPERFIDDLE_ELECTRIC_SERVER_VERSION jvm property must be set in prod?
I've done this for now, but I don't want to have to set this every time:
dokku docker-options:add opentax build '--build-arg VERSION=1911848' (based on my local dirty git version)#2023-08-1110:24Dustin Getzagree this is annoying i am hoping to address it by EOM#2023-08-1110:25Dustin Getzprobably bash script is the best answer today or build.cj#2023-08-1109:44braai engineerIs it expected to see these CLJS build logs during prod build?
...
remote: #25 100.8 <- Compile CLJS: contrib/char$.cljc (115 ms)
remote: #25 100.8 -> Cache write: contrib/char$.cljc
remote: #25 100.9 <- Cache write: contrib/char$.cljc (32 ms)
remote: #25 100.9 -> Compile CLJS: contrib/rfc3986.cljc
remote: #25 100.9 <- Compile CLJS: contrib/rfc3986.cljc (23 ms)
remote: #25 100.9 -> Cache write: contrib/rfc3986.cljc
remote: #25 100.9 <- Cache write: contrib/rfc3986.cljc (20 ms)
...
#2023-08-1110:23Dustin Getzi noticed those logs too and haven’t looked into it#2023-08-1110:25braai engineerAnyone have a .circleci/config.yml lying around for an Electric app? Currently building on my EC2 Dokku box is too slow and building uberjar on my machine and pushing is slow from ZA Internet (we is being Borat 3rd world country, ok)#2023-08-1110:40Dustin Getzfirst of all why circle, dead company, github actions right?#2023-08-1110:41Dustin Getzi don’t know what dokku is#2023-08-1110:41Dustin Getzdo you even need a dockerfile? why not just use git to sync your code to prod#2023-08-1110:41braai engineerCircle is dead?#2023-08-1110:42Dustin Getzcircle is technical debt milk machine#2023-08-1110:42braai engineerthey have really fast build runners. haven't used GH actions in anger tho#2023-08-1110:43braai engineerDokku is a mini PaaS that mimics Heroku: https://dokku.com/
what's nice about it is it has plugins for letsencrypt which makes SSL easier
it does deploy swaps for you
nice if you just have an EC2 box and you want to build and run stuff on the same box#2023-08-1110:44braai engineerso it runs nginx in front of your apps and has CLI for port mapping and domains setup#2023-08-1110:46braai engineereish: https://circleci.com/blog/ceo-jim-rose-email-to-circleci-employees/#2023-08-1113:40braai engineerouch#2023-08-1110:32braai engineerPossible to run Electric Cljs build and Clj build in parallel and then merge the Cljs prod artifact? This is to speed up uberjar prod builds on CircleCI. CircleCI has IMO the best parallel run support. Great for kicking off tests while frontend builds and stop the build early on bug.#2023-08-1110:39Dustin Getzhmm seems possible , you’ll need to refactor build.clj i think#2023-08-1116:46Vincentthat's cool#2023-08-1120:54grounded_sage(.addEventListener folder-el "drop" (fn [_] (file->folder folder-dragged-to file-dragged)))
How do I do a server transaction triggered from an event listener?#2023-08-1207:15Hendrikhave a look at dom/on (without an !). Here is an example usage: https://electric.hyperfiddle.net/user.demo-chat!Chat#2023-08-1207:50grounded_sageYou mean like dom/on “drop”#2023-08-1207:51grounded_sageOn phone. I didn’t try that. I thought dom on was normal handlers and not event listeners. For like click and hover. #2023-08-1207:59HendrikYes. I didn’t try. But dom/on uses e/listen> which uses dom-listener and that calls .addEventListener. So I expect that this should work#2023-08-1208:00grounded_sageOkay. Well that is awesome if I can do that#2023-08-1219:19grounded_sageThis worked seamlessly btw. Really nice#2023-08-1120:59grounded_sageAtm I'm just handling it through shuffling around some data through atoms.#2023-08-1211:12braai engineerWhat should Datomic d/connect retry logic look like for a locally running transactor that takes a while to start?
I ditched Dokku and am now running my uberjar as a SystemD Unit behind nginx (for LetsEncrypt SSL), but my uberjar starts before Datomic Transactor is ready (runs on same machine) despite setting After = network.target datomic.service in my Unit file. Which means I have to manually restart my service after a reboot.#2023-08-1211:41Dustin Getzprobably use missionary , there are examples of looping in missionary try searching for loop or recur#2023-08-1307:20teodorluQuick note: The link to the changelog from https://hyperfiddle.github.io/#/page/64bfd710-51cd-4376-8604-c579a0c231c3 seems to hit a 404. Should it point to https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md instead?#2023-08-1513:20Dustin Getzthanks fixed#2023-08-1308:49teodorluHi! I’m Looking for a bit of advice.
I’m writing a tiny markdown editor with https://pandoc.org/ running on the server. I want to preview the HTML pandoc generated from the markdown in the browser, next to a textarea with markdown in it.
What’s the recommended way to do this from Electric?
At first, thought maybe (dom/div (dom/innerHTML "<p>raw html here</p>")), as with dom/text. But I couldn’t find anything, so perhaps i have to (set! (.-innerHTML el) "<p>raw html</p>"). Attaching a screenshot of where I’m at right now. https://github.com/teodorlu/electric-starter-app/blob/a2087a7cfa9392ccb082f30bf0eb07c904e332c4/src/app/todo_list.cljc#L17-L35.#2023-08-1308:53teodorluI got a preview working with this:
(let [text-html (e/server (-> text pandoc/from-markdown pandoc/to-html))]
(e/client
(-> js/document
(.getElementById "vwnm4o")
(.-innerHTML)
(set! text-html)))
(dom/div
(dom/div (dom/props {:id "vwnm4o"}))
(dom/pre (dom/text (e/server text-html)))
(dom/div
(dom/p (dom/text
"Q: how do I create a dom node with innerHTML equal to html-text?"
" Please advise!"))
(dom/p (dom/text "I'd like to have a preview of HTML I just generated here.")))))
Still very much curious about whether this is a good idea!#2023-08-1309:34teodorluHere’s a (somewhat modified) live version: https://commonmark-edit.app.iterate.no/
(hoping to show some clojure-curious colleagues the nicities of clojure and electric)#2023-08-1314:55Dustin Getzset! on innerHTML is idiomatic#2023-08-1314:56Dustin Getzthat is essentially how dom/text works if you look at the source#2023-08-1315:14teodorluOK, thanks!#2023-08-1311:06HendrikHow can I keep track of that an electric value has changed within the past x seconds?
In pseudo code:
(let [value (from-somewhere)
recently-changed? (What-to-put-here? seconds value)]
For sure I could keep track of the state manually with an atom. However is there a reactive solution?
I think debounce is part of the solution, but I am somewhat stuck:
What I need is a flow that:
• initial returns false
• a debounced subflow that returns true if value changes and after x seconds delay returns false. #2023-08-1312:04Dustin Getzthe atom solution isn’t “bad”, Electric is currently missing direct support for cycles which (1) can be directly emulated with an atom and (2) the atom implementation is a perfectly good low level implementation for a cycle operator. we do publish e/with-cycle which is an experimental macro over this pattern#2023-08-1312:06Dustin Getzperhaps we want to sample the current time every time the value changes (you can use something like (defn sample-system-time-ms [_] System/getTime)), whatever the api is#2023-08-1320:14HendrikThere is no cycle. I finally came up with a working reactive solultion:
(defn changed-recently [flow t]
(let [but-first (m/eduction (comp (drop 1) (map (constantly true))) flow)]
(m/reductions {}
false
(gather
but-first
(m/ap (let [x (m/?< but-first)]
(try (m/? (m/sleep t false))
(catch Cancelled _ (m/amb)))))))))
;Usage
(new (changed-recently (e/fn [] some-value) 1000))
I am a bit concerned about using gather, which has been deprecated.#2023-08-1418:04Dustin Getzyou mean gather was removed from missionary? i think leo just didn’t want dozens of utility primitives in missionary, you can PR it to electric missionary contrib ns, which will be standalone someday and Leo reviewed#2023-08-1418:07Dustin GetzAs to the original q I envision a pure function that takes (current value, current time, previous value, previous time) where the previous state is fed back into the function via recursion, which is a cycle. Using actual electric recursion will stackoverflow iiuc (we have work to do here) so accomplishing the feedback with an atom instead is appropriate#2023-08-1418:08Dustin Getze/with-cycle is an experimental macro over this pattern#2023-08-1418:13Dustin Getzyour discrete time solution seems fine based on a quick review on mobile #2023-08-1319:21AmosHi, does anyone try to use frontend db like (datascript) with electric. I try to do that and I find difficulty to iterate the frontend data reactively since I can’t use the (e/for).#2023-08-1406:49xificurCyou can e/for on the client. Typically we loop over server data which is why most demos will show that#2023-08-1408:35AmosThank you.#2023-08-1412:56braai engineerNot Electric-related but for my Electric app: how are guys deploying uberjars in the wild? At the moment, I'm running a GH build runner on my prod machine, on successful build, delete old uberjar, download build artifact and cycling SystemD service.
Without going full K8s, is there a nicer blue/green deployment strategy that plays nicely with Clojure apps? cc @robert-stuttaford are you using Immutant perhaps?#2023-08-1413:57nakkayaI build a docker image on the CI and deploy that. Develop on a branch where CI only runs the unit tests then when merged into master CI will build a docker image, SSH into my prod machine pull the image and restart the container.#2023-08-1414:14braai engineer(btw. Java / Clojure is already virtualized – don't need Docker. I ditched it - less complexity)#2023-08-1414:17frozenlockI'm doing pretty much the same: uberjar, overwrite the jar on prod server, restart service. Simple, fast, efficient.#2023-08-1414:17frozenlockI've worked with teams that would waste weeks/months of developer time building a deployment chain... "chaining" themselves to a particular ecosystem.#2023-08-1414:18braai engineerKnow any tricks to speed up uberjar build? My (relatively simple) Electric app takes 4 minutes to build on GH actions standard-sized box. Parallelising cljs & clj is the natural start.#2023-08-1414:18frozenlockNo sorry. Currently all my prod apps are Clojure only or built with Reagent in the case of SPAs. I hope to go deeper in Electric soon.#2023-08-1507:26robert-stuttafordour production build. we use buildkite and our own babashka scripts to orchestrate everything. ">" in this listing means 'wait for everything to the left before starting the stuff on the right':#2023-08-1507:27braai engineer@robert-stuttaford haven't used Buildkite. Any advantages over GH Actions? (I'm principally wary of putting too many eggs in one basket with GH)#2023-08-1507:28robert-stuttafordits agents run on our EC2 instances, so we can control perf and keep everything inside our security cordon#2023-08-1507:29braai engineerbtw. GH Actions has self-hosted runners#2023-08-1507:29robert-stuttafordwe built this system before GHA was a thing 🙂#2023-08-1507:30robert-stuttafordwe only use GH for source hosting, we don't really use PRs or any of that jazz#2023-08-1507:31robert-stuttafordhttps://trunkbaseddevelopment.com/ baby#2023-08-1507:31braai engineerjunk in da trunkkkk#2023-08-1507:32braai engineer@robert-stuttaford have you seen Pijul? https://pijul.org/#2023-08-1507:33braai engineerJujutsu has similar first-class conflicts https://github.com/martinvonz/jj#2023-08-1507:34robert-stuttafordjj looks neat#2023-08-1507:35braai engineerI'm holding out for a Lisp-syntax aware VCS that has first-class conflict resolution so whitespace changes do not add noise. Just let the client format their code maybe with metadata about how they want it formatted.#2023-08-1419:34braai engineerHow to support multiple manifests / frontends in one Electric project? I want to split my app in two: thin view for anon clients and thicc view for admins, but same project. I could make a separate project, but different ports and other annoyances.
I've added another shadow-cljs build, which compiles, but index.html module templating is the same and I see admin namespaces in the "client" manifest. Ideally I would like the client connection be keyed on namespace or build ID and then I only get the AST stuff I need for a particular frontend.
In meantime will probably combine the two and do different routing for the time being with an /admin route.#2023-08-1516:35Dustin Getzfrom this perspective Electric apps are ordinary shadow-cljs apps#2023-08-1516:35Dustin Getzso check your :require graph#2023-08-1517:57Dustin GetzRed Planet Labs announcement in 3 minutes!#2023-08-1518:13frozenlockI must have missed something. How is Red Planet Labs related to Electric?#2023-08-1518:15Dustin Getzno business relationship; the technology allows making reactive backends which Electric can natively consume#2023-08-1518:37Dustin GetzI summarized here : https://www.reddit.com/r/Clojure/comments/15rzyqq/how_we_reduced_the_cost_of_building_twitter_at/#2023-08-1518:38JNot a word of Clojure but when I read the blog post Clojure philosophy is here.#2023-08-1520:27Daniel JompheOf particular interest for Hyperfiddle seems to be this part of the conclusion:
> I didn’t mention “fine-grained reactivity”, a new capability provided by Rama that’s never existed before. It allows for true incremental reactivity from the backend up through the frontend. Among other things it will enable UI frameworks to be fully incremental instead of doing expensive diffs to find out what changed.#2023-08-1520:29Daniel JompheThis true incremental reactivity won’t need to be sent to the frontend by Rama. Obviously, Hyperfiddle can subscribe to it and color whatever reactions for the server or the client.#2023-08-1523:02bhurlow> Will there be a first-party Clojure wrapper? Or, would the expectation be that users would use Java interop?
> answer: We’re not releasing one as we don’t have the bandwidth right now to maintain and document another API. That said, making a Clojure wrapper around the Java API should be pretty easy.#2023-08-1523:02bhurlow^ from Nathan in the HN post#2023-08-1523:06Dustin Getzyeah i mean look closely at the java “not a DSL” DSL, it is … unclear if java programmers will accept that as java#2023-08-1523:07Dustin Getzi wonder what their clojure macro DSL is/was#2023-08-1604:18tatutI have never needed “twitter scale” in anything I’ve done, so it is not clear how this would 100x my productivity in a typical app… but the proof is in the pudding, will definitely need to play around with this#2023-08-1605:31Ben SlessI probably haven't written anything twitter scale but pretty high scale
So many things can go awry, the breakdown to micro services which should help you be reactive and encapsulate difficulties multiplies the surface area of the problem
Databases, monitoring, health, logging, queueing systems, man, gimme Rama now if I can cut all that crap out#2023-08-1612:40Daniel JompheTheir Java API is a thin wrapper over their actual Clojure API, which they’re https://groups.google.com/g/rama-user/c/aNZds6mbnwE.
I suppose they don’t want anyone in the community to be able to claim that Rama is better for clojure users, because it would hurt their true adoption target: the world of enterprise Java apps.
Knowing that, It’d feel wrong to me to produce a Clojure wrapper over the Java API, which is itself a wrapper over the internal Clojure API. But still, if it could mean I can develop Rama with my REPL and plain clojure functions, yeah, I suppose I’d still compromise. 🙂#2023-08-1613:06bhurlowAlso, it seemed from the HN thread that the Clojure DSL was used internally for implementation. Not clear if that is the same as the userland API. The Java DSL as it currently stands looks like a big adoption hurdle to me#2023-08-1615:18marciolTalking to @U2J4FRT2T he said that the DSL resembles Specter, and it can related to what @U09K620SG said some time ago:
https://old.reddit.com/r/Clojure/comments/izh9o4/specter_vs_meander_vs_handcrafted_code_when/g6jrq48/#2023-08-1615:19marciolTo make this more approachable to Java devs, maybe wrapping it into a Java Stream compliant API would help.#2023-08-1615:23Dustin GetzThanks for linking that I would love to see higher level database-like abstractions built on top of Rama (however the closed source nature means that this will not happen)#2023-08-1615:25Dustin Getzimagining a streaming Datomic-like database with datalog, transactions etc that lets you drop down to the "Rama layer" to code queries directly against custom indexes#2023-08-1615:26Dustin GetzThat's really the main barrier to Rama adoption that I see – if it is really a tool for database engineers to develop scalable custom indexes, that is a fairly narrow use case#2023-08-1615:26marciolYep, it would be awesome#2023-08-1615:44Ben SlessDoes it handle data loops?
If so, you can compile datalog to it
Another thing I didn't see yet is how data stores get the data in them. If you can feed ETLs back to those you have a stream processing platform not just a db#2023-08-1615:57Dustin Getzyou mean "depot appends" ? I think it's like depotReference.append(record)#2023-08-1719:00Daniel JompheNathan didn’t confirm what I wrote in all the details, but graciously provided some interesting bits:
> I suppose I shouldn’t be surprised that Rama would resonate more immediately with Clojure programmers, given the emphasis on data structures and its origins in functional programming. So exposing the Clojure API may come sooner rather than later – no promises though. I’m curious how everyone’s experience is next week using Rama through Java interop.
>
> We did feel the pain of not having a proper REPL while developing Mastodon in straight Java, which brought me back to my pre-Clojure days. It’s annoying to have to reload everything from scratch when wanting to test a small change. We did a little bit of work to try to be able to reload Java classes dynamically, but we never could get it working robustly especially when there were changes across multiple files. You won’t have any of these issues developing modules from Clojure with Java interop.
https://groups.google.com/g/rama-user/c/aNZds6mbnwE/m/bCpUdFhVBAAJ#2023-09-1320:36bhurlowSpoke with Natahan today, our company is going to be experimenting with the beta. I asked for feedback on what’s ok to share, but in response to @UK0810AQ2’s question above, using datalog with rama isn’t required or desired, the idea is that targeted data structures with associated indexing is better than an arbitrary query language – It’s not normalized by design#2023-09-1406:53xificurCask for opening the clojure API 😉#2023-09-1418:01bhurlowIt’s coming, we reviewed it and its real. It’s pretty heavy on the DSL though, think similar to core.async. It will take some time to adapt the mental model#2023-08-1520:27Daniel JompheOf particular interest for Hyperfiddle seems to be this part of the conclusion:
> I didn’t mention “fine-grained reactivity”, a new capability provided by Rama that’s never existed before. It allows for true incremental reactivity from the backend up through the frontend. Among other things it will enable UI frameworks to be fully incremental instead of doing expensive diffs to find out what changed.#2023-08-1616:35braai engineerHmm, how do I debug this error? I looked around for a client DOM effect inside an e/server but I don't see any.
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Encountered error when macroexpanding hyperfiddle.electric/boot.
Unable to resolve symbol: hyperfiddle.electric-dom2/new-node
{:file "myproject/core.cljc", :in [hyperfiddle.electric-dom2/new-node]}
ExceptionInfo: Unable to resolve symbol: hyperfiddle.electric-dom2/new-node
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:647)
hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:632)
hyperfiddle.electric.impl.compiler/eval11125/analyze--11131/fn--11136/fn--11137 (compiler.clj:700)
hyperfiddle.electric.impl.compiler/eval11125/analyze--11131/fn--11136 (compiler.clj:699)
hyperfiddle.electric.impl.compiler/eval11125/analyze--11131 (compiler.clj:688)
hyperfiddle.electric/boot (electric.cljc:378)
hyperfiddle.electric/boot (electric.cljc:373)
clojure.core/apply (core.clj:671)
clojure.core/apply (core.clj:662)#2023-08-1616:37braai engineerSolved. I had,
(e/defn SomeView []
(dom/div ...))
;; lower down:
(e/server (SomeView.)
And was calling it from an e/server block. Fixed with this:
(e/defn SomeView []
(e/client
(dom/div ...)))#2023-08-1616:37braai engineerI hope incremental will be able to show which e/fn is the culprit :)#2023-08-1616:56braai engineerIs there an example of how to use Missionary from Electric? I don't see one here, but may be missing it: https://github.com/hyperfiddle/electric/tree/0a63292a9baa7783c7f647e4cfffeaf5055e9f64/src-docs/user/missionary
e.g. how to sleep from Electric with m/sleep.#2023-08-1617:42Dustin Getzsearch m/sleep it's been discussed#2023-08-1617:07telekidBit of a shower thought – electric's semantics are more or less exactly what you want out of a polyglot build tool like bazel#2023-08-1617:08telekid(the major gap that I see being that the state of electric's nodes obviously don't persist between process restarts)#2023-08-1617:09telekidotherwise, electric is more or less exactly what I'd want out of a build system#2023-08-1617:40Dustin Getznot familiar with bazel but yes agree, "effect system" structure unifies a huge amount of infrastructure that today is disjoint#2023-08-1617:40Dustin GetzI maintain a list:#2023-08-1617:40Dustin Getz• hot take: the following technologies unify under the umbrella concept of "functional effect system"
◦ Kubernetes
◦ Erlang process supervision
◦ Microsoft Orleans
◦ Akka
◦ Rx
◦ Zio
◦ Cloud Haskell
◦ Clojure's Missionary
◦ Electric Clojure
◦ Unix, Windows (process/program/task manager)
◦ Java threads, OS threads
◦ Frontend rendering with signals
#2023-08-1617:40Dustin Getz• value prop — functional effect systems
◦ cancellation
◦ concurrency and backpressure combinators
◦ composability / referential transparency – coordinate large scale effect fabrics
◦ glitch-free event propagation
◦ correct error handling by default (process supervision)
◦ resource lifecycle (RAII) – strong resource cleanup guarantees
#2023-08-1617:41Dustin Getz• List of relevant topics:
◦ Process supervision (trees)
◦ Observable sharing (DAGs)
◦ Graceful termination
◦ Self-healing software
◦ Resilience
◦ Actors
◦ Signals and event streams / FRP
◦ functional effects
◦ concurrency and backpressure
#2023-08-2013:53teodorlu> Bit of a shower thought – electric's semantics are more or less exactly what you want out of a polyglot build tool like bazel
>
Or the semantics for a "thing to manage your system" like integrant or component. Feels like integrant solves the narrow, parochial version, whereas electric solves the general version.
On that sidenote, JVM only Electric could make sense - for cases where one wants the reactivity, but don't want reactive web clients.#2023-08-2013:56Garrett Hopper> JVM only Electric could make sense
CLJS only could be useful as well for that matter.
Essential allowing Electric reactivity without requiring the websocket connection. (And then disallowing e/server & e/client when there is no connection)#2023-08-2014:05Dustin GetzJVM and JS only i think already works (just don't use e/client e/server) – the e/run unit test entrypoint mocks out the network. However the runtime overhead for network management is still present (may not matter)#2023-08-2014:06Dustin GetzIndeed electric dynamic scope is roughly competitive with Component and other DI frameworks#2023-08-2014:08Dustin GetzWe don't advertise it because we have not done like a requirements analysis to figure out exactly what we have and don't have for a general purpose DI use case like that ... we're super focused on web UI. Using Electric for a different use case likely will have rough edges or some mismatches with how Clojure programmers might be accustomed to structuring your systems, we don't know yet#2023-08-2014:27teodorluRight—to get something as nice as Integrant for local development with Electric definitely requires some thinking and some work.#2023-08-1711:04joshchoCurious if there are interesting ideas for in-browser devtools like re-frisk or Fulcro inspect that leverage Electric's unique powers. My own, but maybe sth like https://clojurians.slack.com/archives/C7Q9GSHFV/p1691215465636979 as a chrome extension could be cool.#2023-08-1715:25Dustin Getzabsolutely possible, sky is the limit to introspect and reflect the DAG and in depth observability#2023-08-1912:40Dustin GetzHere is the Electric "DAG bytecode" – https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_compiler_internals.cljc#2023-08-1912:41Dustin GetzI think it would be straightforward to visualize that using GraphViz or whatever#2023-08-1912:42Dustin GetzThat would be step 1; step 2 would be to observe the events on the wire protocol in order to animate the graph and see values flow through it#2023-08-1912:43Dustin GetzSeparate idea – a "reactive REPL"#2023-08-1912:43Dustin Getzin a simple terminal repl, you type a form and press enter and get a result. Request/response#2023-08-1912:45Dustin GetzImagine that same terminal UI except in a browser and the response is animated like the Two Clocks demo#2023-08-1912:46Dustin Getzuser=> (println (e/server e/system-time-ms))
1692449153122 ; animated#2023-08-1912:46Dustin Getzeach REPL interaction is a live app and stays live until you delete it#2023-08-1912:47Dustin Getzthe blocker to this is that Electric is still a compiler (no eval) so you have to wait on shadow rebuilds#2023-08-1912:48Dustin Getzin 2024 Electric will become a JIT (like the JVM and also Clojure), moving the compiler analysis to runtime, which will enable such a REPL without iframing a second shadow instance or something#2023-08-1912:49Dustin Getzthe live-coding demo video on our homepage http://www.hyperfiddle.net uses a second shadow instance with an iframe (and the video is also sped up to hide the hot code reloading pauses)#2023-08-1912:50Dustin GetzA cheaper alternative is making Electric compatible with SCI, which has no blockers but someone has to do the work it's probably a few weeks of work that we dont have time for, perhaps someone could get Clojurists Together funding for this#2023-08-1913:11Dustin GetzFinally we still are looking for notebook infrastructure a la NextJournal that is unbundled and separate from any application level concerns. "Session-lived backend" is the term for an on-demand "flyweight" backend to spin up and host a throwaway JVM and classpath for livecoding "fiddle" infrastructure. This missing piece gets end-user programmers the ability to build real apps (that have dependencies and need classpath control, i.e. can safely connect to some arbitrary database from userland)#2023-08-1720:59Garrett HopperHas anyone tried or considered running an Electric server on Datomic Ions?
Not sure how it would work due to the inherent stateful relationship of websocket <-> server instance.
Would be very interesting to have the Electric server running directly in the DB though :thinking_face:#2023-08-1721:40Dustin Getzwe have only run it so far in onprem, there are some threads discussing the websocket issue if you search this channel and #C03RZMDSH #2023-08-1721:43Garrett Hopper👍 I searched for Ions and only found a brief mention from 2018; will look more for websockets 👀#2023-08-1721:43Dustin Getzsearch by @U0514DPR7 they are his threads#2023-08-1721:44Garrett HopperThank you 🙂#2023-08-1721:59Garrett HopperSeems like it's probably possible through API Gateway's websocket support.
Just need to abstract the session management outside of the electric adapter.
Is there any documentation on different adapters?
Seems like I could reuse most of the logic from hyperfiddle.electric-jetty-adapter?#2023-08-1722:52Dustin Getzcross linking other thread https://clojurians.slack.com/archives/C03RZMDSH/p1692310904023429#2023-08-1723:25Daniel Jomphehttps://clojurians.slack.com/archives/CL85MBPEF/p1676057124117659?thread_ts=1675723956.514279&cid=CL85MBPEF.#2023-08-1812:51Dustin Getz@U5JUDH2UE if you would summarize the issue please in as few bullet points as possible, that would be helpful#2023-08-1813:46Garrett Hopper• Electric requires the WS session to be handled by the same server instance for the duration of a user's session due to its inherent stateful nature. (Running the server side version of components and sending the client any data when the client expects to receive it.)
• Diatomic Ions are useful for deploying Clojure functions in a scalable way while also having very fast Diatomic DB access.
• Ions can be exposed either through a Lamda proxy or API Gateway HTTP endpoint.
• There's no way to route either to always be handled by the same server instance when scaled horizontally.
• This could be handled by placing WS events in something like Redis and relying on each instance to publish messages it received and subscribe to messages related to sessions it's currently serving.
• Doing so would also require a custom Electric adapter which abstracts the API Gateway WS abstraction where new events being received are translated to HTTP requests and sending events requires calling an HTTP endpoint.
• Ions also don't seem to provide any lifecycle hooks when being scaled down, which would be a major problem for long-lived Electric sessions which would be lost and require a user to refresh/connect elsewhere.
All together, I'm not sure Ions are the right deployment solution given Electric's design. The server/client needing to run the same code in parallel and communicating throughout is, I think, inherently different from what Ions expect and likely requires a more traditional server deployment.#2023-08-1813:47Garrett HopperThat's not very few 😬
More of the story is that it probably shouldn't be done, I think.#2023-08-1815:04Dustin GetzThanks, I'd guess the path forward is to reject that sticky session routing to an ion is impossible and figure out how to do it#2023-08-1815:05Dustin Getzanyway isn't this a dead product? Nubank uses Datomic Onprem#2023-08-1815:06Garrett HopperI think you might be right about that. I'd never looked closely, but I was getting a sense that it may not be desirable to use even outside of this limitation.#2023-08-1821:32Daniel JompheDustin… you’re… saying it… out loud…
I’ve been “regularly” asking them if they’ve started to use Datomic Cloud at Nubank. Never got a response. So: they didn’t yet start to use it.
Still, the way they develop first for On-Prem, but in a Cloud-compatible way, then release things for Datomic Cloud a few months later, proves that the product is not yet dead at all in their mind. I don’t know if it’s just to support their existing Cloud clients, or if they’re really investing in it for the long run. I’ve got the feeling they want the long run for it, but might someday unify things so much so that the CloudFormation templates for configuring AWS would also become useful for On-Prem (Clouded On-Prem).#2023-09-0119:51stuarthallowayThe "release for Cloud later" was mostly an artifact of jumping through hoops of the AWS marketplace. Now that Datomic Cloud is free, no marketplace hoops, so we should be able to release Cloud and Pro at more nearly the same time.#2023-09-0119:51stuarthallowayThe "release for Cloud later" was mostly an artifact of jumping through hoops of the AWS marketplace. Now that Datomic Cloud is free, no marketplace hoops, so we should be able to release Cloud and Pro at more nearly the same time.#2023-08-1810:16Adrian SmithA friend of mine is experiencing unexpected behaviour with electric
I've minimised his project to help isolate problems and the problem seems to be around e/offload
If you take the xtdb-starter project then paste this file into it: https://gist.github.com/slifin/5ef1774047e516efe11ef249b3ad1e9a
Then click new record, and then really spam and go to town on the "increment" button you will see sometimes that the two numbers go out of sync
It seems like e/offload is executing its body more times than it should :thinking_face:#2023-08-1810:17Adrian Smith#2023-08-1810:22Dustin Getzi’ll take a look, quickly which electric version? e/offload had fixes#2023-08-1812:28Dustin GetzThis is a cycle on e due to a subtle race condition in dom/on which is used by ui4/button#2023-08-1812:28Dustin Getzthe workaround today is to use e/snapshot to break the cycle on e#2023-08-1812:29Dustin GetzThe root cause is that dom/on sucks and we're replacing it with something better in ui5#2023-08-1812:30Dustin GetzIn the meantime, we're stuck with dom/on which does at least have fully determined semantics and we believe no bugs, you just have to understand the semantics to use it, which require you to think in DAGs#2023-08-1812:32Dustin GetzSome recent discussions:#2023-08-1812:34Dustin Getz• https://clojurians.slack.com/archives/C7Q9GSHFV/p1691772416156519?thread_ts=1691740330.761899&cid=C7Q9GSHFV
• https://clojurians.slack.com/archives/C7Q9GSHFV/p1691418778487929?thread_ts=1691418519.535529&cid=C7Q9GSHFV
• https://clojurians.slack.com/archives/C7Q9GSHFV/p1691412723234059?thread_ts=1691393945.636769&cid=C7Q9GSHFV#2023-08-1812:35Dustin GetzPlease apologize for me to your friend for making him need to understand this#2023-08-1813:01Adrian SmithI will, thank you for the context#2023-08-1812:26Dustin GetzRing is getting websocket support, please comment if this is relevant to you, I am working on build and deployment this week#2023-08-1812:27Dustin Getzhttps://github.com/ring-clojure/ring/blob/master/CHANGELOG.md#1110-alpha1-2023-08-17#2023-08-1813:55nivekuilif this means we can easily drop electric into a project as just a ring handler then yes#2023-08-1813:59Garrett Hopperhttps://github.com/ring-clojure/ring/blob/2.0/SPEC-2.md#3-websockets#2023-08-1814:00Garrett HopperSeems like it would make the existing server specific Electric adapters much cleaner. 😁#2023-08-1814:44weiwhat's a good way to block (on the server) until an atom changes to a specific value? I suspect there's an appropriate missionary construct but I'm not yet familiar with it#2023-08-1814:45Dustin Getzplease elaborate?#2023-08-1814:48weie.g. in a payment flow, I'd like to pop up a payment request, then wait for the client to pay after which the server is notified via webhook. then the server does some more stuff and returns the result to the client. it's the "wait on the webhook" part I wasn't sure how to implement in electric#2023-08-1814:49Dustin GetzIs this an electric question?#2023-08-1814:49weiI was thinking of updating an atom when that happens since a lot of the electric demo apps use atoms. but I didn't find examples that blocked on the state of an atom#2023-08-1814:49weinot sure.. maybe a missionary question?#2023-08-1814:50weii can ask elsewhere#2023-08-1814:50Dustin Getzby "block" you mean "run effect when atom has certain value" ?#2023-08-1814:51Dustin Getz(when (= 42 (e/watch !x)) (println "hi"))#2023-08-1814:52Dustin GetzElectric is a non-blocking computation so the question isn't type checking in my mind#2023-08-1814:53weii see, i guess I'm trying to implement this payment flow in electric, which to me is a series of blocking steps. still trying to wrap my head around electric.#2023-08-1814:54weiyour example is simple enough though, will play around with it#2023-08-1814:55Dustin Getzyou can bridge the webhook into electric directly with m/observe (instead of an atom), if you search there are several discussions#2023-08-1815:00weifound the discussions and will look into it, thanks!#2023-08-1815:01weifinding that building in electric requires unlearning a lot of crufty habits, which is refreshing#2023-08-1901:30Vincent(e/client (payment stuff) (e/server (let [webhook-status (...)] (e/client (when (= webhook-status :pait) ...)))))#2023-08-1901:30Vincentpotentially#2023-08-1907:20weivery helpful, thanks! mind elaborating on what goes into (let [webhook-status (...)])? do i have to do something in my webhook handler? sorry if that's an elementary question. i read the m/observe docstring and didn't understand it.#2023-08-1907:21weialso, do i have to use e/offload since it may be a long time before the user completes the payment and the webhook is triggered?#2023-08-1911:47Dustin Getze/offload is about using a threadpool to turn a blocking computation into a non-blocking computation#2023-08-1911:49Dustin Getzm/observe is about adapting a callback (your webhook) into a discrete flow#2023-08-1911:50Dustin GetzIt may be easier to just write your backend workflows in Clojure and put the result into an atom/database and use the atom/database to signal state changes to the UI#2023-08-1911:51Dustin GetzCertainly workflows can be done natively in Electric and/or Missionary but the idioms aren't fully worked out yet, so perhaps not the right place to start learning Electric/Missionary#2023-08-1902:42Garrett HopperIs it possible to call hyperfiddle.electric/boot with a dynamic e/defn?
(e/defn Example [] (dom/text "test"))
(def component #'Example)
(electric/boot
(binding [dom/node el]
(new component)))
I can't seem to get a var to work as expected. (Despite it having the same electric compiler metadata on it)
(My use case is attempting to implement a https://github.com/cjohansen/portfolio defscene macro which will need to dynamically create the e/defn for each scene.)#2023-08-1902:47Garrett HopperI guess the problem would be that these are not quite equal.
And Electric doesn't appear to support rendering a var that points to another (electric) var.
(def ^{:test true} test {})
(def indirect-test #'test)
(= (meta #'test)
(meta indirect-test))#2023-08-1910:27Dustin Getzyeah the indirection won’t be resolved, we will need to think about this. #2023-08-1915:45Dustin GetzCan portfolio be changed? Really don't love this#2023-08-1915:46Dustin Getzalso this is trouble : "dynamically create the e/defn for each scene"#2023-08-1915:47Dustin GetzElectric today is a straightforward compiler, runtime eval is not a thing#2023-08-1915:47Dustin GetzWe do need eval (Hyperfiddle/HFQL needs eval) and it is probably our #1 goal for next spring#2023-08-1915:54Garrett HopperEer, the defn isn't dynamic, but it is creating indirectly through a macro.
I think I was able to workaround it by making that macro expansion also include the boot command. 🤞#2023-08-2009:12braai engineersuhhhh excited for incremental Electric#2023-08-2013:49teodorluYessssssss#2023-08-2013:49teodorluCurrently the main drawback of electric compared to other options IMHO. At least from the point of view of local development experience.#2023-08-2114:12nakkayaOn one of my forms I started getting Method code too large! during uberjar build. I tried splitting the form into multiple e/defn s but this did not help. Is there workaround for this?#2023-08-2114:12Dustin GetzCan you add me to your repo#2023-08-2114:17nakkayaI never removed you 🙂 you should still have access. Culprit seems to be [1] and the CI failure is at [2]. Commenting out selects fixes the issue.
[1] https://github.com/nakkaya/dentmetria/blob/main/src/app/core.cljc#L514
[2] https://github.com/nakkaya/dentmetria/actions/runs/5896358653/job/15994411034#2023-08-2114:17Dustin Getzcommenting out which lines?#2023-08-2114:18nakkayanot lines but whole expressions, two dom/select s in the form.#2023-08-2114:18nakkayaon 541 & 578#2023-08-2114:39Dustin GetzOk, i looked - you have about 700 LOC electric, is that right?#2023-08-2114:39Dustin GetzWe have electric apps with a lot more code than that, but haven't seen this error#2023-08-2114:39Dustin GetzWe will need to investigate, but it means there should be some sort of workaround until the next compiler lands#2023-08-2114:45nakkaya> Ok, i looked - you have about 700 LOC electric, is that right?
Yeap, I also have in the same file a much larger form that does not cause any trouble.#2023-08-2208:31braai engineerOne of my uberjar builds is also blocked on this. @U5H4U2HEH temp. workaround is deploy a dev build to prod (not ideal).#2023-08-2211:40Dustin GetzPossible workarounds
1. disable AOT compilation in the uberjar, only the client optimizations matter
2. bypass uberjar entirely, launch prod with clj, i.e.:
a. instead of java -cp app.jar -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$VERSION clojure.main -m prod
b. use clj -J-DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$VERSION -A:prod -M -m user
i. not sure if this is exactly the right Clojure CLI incantation#2023-08-2211:50Dustin Getzin both cases the point is to bypass Clojure's AOT (ahead-of-time) compilation, which is not important. As a recap, Clojure-JVM itself is a compiler (it compiles Clojure code to java bytecode in .class files), generally Clojure-JVM compilation happens on startup as each namespace is loaded (both in your app and in your dependencies). For large Clojure-JVM projects (100k LOC) this causes long startup times so the AOT option lets you ship precompiled .class files in the jar.#2023-08-2211:51Dustin GetzAOT compilation is also important in libraries like Datomic where you don't want to ship Clojure source code in the .jar#2023-08-2221:24nakkayaI went with option 1, I was able to deploy the jar.#2023-08-2222:24Dustin Getzyou skipped AOT and ended up with a working uberjar?#2023-08-2307:35nakkayayeap#2023-08-2220:22VincentI was thinking to put an animated gif while the app is loading but i'm not sure how to achieve that#2023-08-2220:59Dustin GetzHow long are your load times? Pending is still sort of a mess, so try/catch Pending might not give you what you want#2023-08-2318:12Vincentnot very long at all. sub 2-3 seconds#2023-08-2221:40Garrett HopperIf anyone has interest in using Storybook to develop isolated Electric components: https://github.com/garrett-hopper/ElectricStorybookExamples#2023-08-2222:26Dustin Getzawesome#2023-08-2308:55Stef CoetzeeIs it par for the course to define CLJS deps in deps.edn a.o.t. shadow-cljs.edn in full-stack projects like Electric?#2023-08-2309:16Stef CoetzeeAt least if using Calva, yes:
“Configure shadow-cljs to use deps.edn for source-paths and dependencies.”
https://blog.agical.se/en/posts/shadow-cljs-clojure-cljurescript-calva-nrepl-basics/#2023-08-2300:13Dustin Getzhere is the rama social network demo https://github.com/redplanetlabs/rama-examples/blob/master/src/main/java/rama/examples/ramaspace/RamaSpaceModule.java#2023-08-2303:36joshchotwitter thread with video: https://x.com/nathanmarz/status/1694046536861368594?s=61&t=lI9nNO9bkbL1YKFrdIw_Tg#2023-08-2305:10Stef CoetzeeVideo only:
https://www.youtube.com/watch?v=jcvxGvzjLHg#2023-08-2306:41henrikHopefully they’ll relent and give access to the Clojure API, because building a bespoke Clojure API around a Java API around a Clojure API is silly.#2023-08-2310:18danielstocktonI'm new to following this stuff. Can anyone tell me how Rama relates to electric? Is it another take on the idea of combining front and backend, or somehow complimentary?#2023-08-2310:22danielstocktonIt looks like it's a paradigm similar to differential dataflow?#2023-08-2310:40Dustin Getzit is complimentary, rama data tier to electric UI tier is a native fit#2023-08-2310:40Dustin Getzin theory at least#2023-08-2310:41danielstocktonAm i right that its a framework for streaming incremental updates to data similar to differential dataflow then?#2023-08-2310:43Dustin Getzi don’t know if rama “subindexing” is actually differential dataflow or not (i doubt it and i think that’s probably a mistake). I can say that the next major version of Electric is indeed differential dataflow#2023-08-2310:43Dustin Getzcertainly rama is similar to and competitive w DD#2023-08-2310:59mmerIt is no coincident that when you see the Paths in the demo that they look like Specter, because they are both supported by the team.
I would think this could mean that a RAMA Clojure implementation could be on the cards.#2023-08-2311:01henrikRama is written in Clojure, so it already exists. The “native” API isn’t exposed though.#2023-08-2311:03mmerI was just looking at the video where it is all Java, seeing the groovy examples, immediately makes you want to usea Clojure variant as it would be much simpler.#2023-08-2311:03henrikLike, they call $$… “vars”. Probably because they are vars in the Clojure API language.#2023-08-2311:08mmerThe code that is written in their examples, is really hard to understand as you are using one language to write another, I would image that macros, could be created to make it simpler to create the code more directly.
Having worked in solutions where you are generating code from code, you find the debugging the emitted code is complex as you don't have the support of IDE etc.#2023-08-2311:09henrik@dustingetz Do you imagine subindexing being DD or not would surface in the API, or more of a mistake regarding the efficiency of the internals?#2023-08-2311:11henrikThe examples certainly do some contortions due to (I’m guessing) the limitations of Java. Variables being strings beginning with */`$$`, and if you want a literal string with those prefixes you must wrap them in Constant comes to mind.#2023-08-2311:19henrikFrom #rama: https://github.com/dundalek/rama-clj-playground#2023-08-2311:33Ben SlessI stumbled on a line where they mentioned the importance of ordering, which is a prerequisite for timely dataflow#2023-08-2311:44Dustin Getzthe differential dataflow end user programming model, as I understand it and as we are implementing it in Electric, is FRP like and DAG based (in Electric you build your app out of lambdas and closures). As opposed to the Rama end user programming model where you program with structs and you have to plan out the data structures. My initial impression is Rama feels like intrusive data structures in C as opposed to generic data structures in Java. Faster but lower level, used in operating systems and games. https://stackoverflow.com/questions/5004162/what-does-it-mean-for-a-data-structure-to-be-intrusive#2023-08-2314:14Ben SlessDD exists at a different abstraction level, too. It's about sets (signals? maybe), not about a specific value or a specific time#2023-08-2307:48KeinWhat is recommended to NOT update a value if the input is under some condition? Do we have to save the previous value in a state?#2023-08-2308:28xificurCcan you be more specific?#2023-08-2308:33KeinWhen syncing text data from server to client, we need to keep track of whether the server’s data is outdated by comparing versions. If the incoming data is outdated, then the client state won’t be updated. To prevent a function from being updated in electric context , we need to calculate the previous value. Is there a recommended way to achieve this ?#2023-08-2308:40KeinSay we have:
(e/defn my-func [a b] (+ a b) )
In state1 , a=1, b=1, result = 2
In state2 , a=2, b=1, result = 3
Can we block state2's update under some condition, and let the result remain 2? Such as :
(e/defn my-func [a b] (if block-condition previous-value (+ a b) )
#2023-08-2308:54xificurC> When syncing text data from server to client, we need to keep track of whether the server’s data is outdated by comparing versions
How does this situation arise? Electric doesn't produce out-of-order data. Is your server sending old data to the UI?#2023-08-2308:56xificurCto also answer the broader question (how to act on previous values), it is typical to introduce state here (an atom)
(let [!v (atom 0)]
(when condition (reset! !v (+ a b)))
(e/watch !v))
#2023-08-2309:11Kein> How does this situation arise? Electric doesn’t produce out-of-order data. Is your server sending old data to the UI?
The outdatedness is defined in our application domain, not in system level. We try to implement an offline-first model where there are n+1 copies of data. And we annotate different copies by version and event-time to resolve conflict.#2023-08-2309:12Kein> to also answer the broader question (how to act on previous values), it is typical to introduce state here (an atom)
Ah I see. Thanks for showing this method.#2023-08-2312:09Dustin Getznote that state and recursion are deeply related and/or the same thing. Consider a Java for loop with mutable accumulator vs reduce#2023-08-2312:11Dustin GetzSo, I hope next year we can eliminate the need for an atom here, but it is not a priority, and i don't find the atom to be problematic for this reason#2023-08-2312:12KeinOkay, I see your point#2023-08-2312:16danbuneaI am trying to run the electric-datomic-browser, but I can't get shadow-cljs to compile the front
clj -A:dev -X user/main
Starting Electric compiler and server...
shadow-cljs - server version: 2.25.3 running at
shadow-cljs - nREPL server started on port 9001
[:dev] Configuring build.
[:dev] Compiling ...
[:dev] Build failure:
------ ERROR -------------------------------------------------------------------
File: jar:file:/Users/danbunea/.m2/repository/thheller/shadow-cljs/2.25.3/shadow-cljs-2.25.3.jar!/shadow/cljs/devtools/client/hud.cljs:1:1
--------------------------------------------------------------------------------
1 | (ns shadow.cljs.devtools.client.hud
-------^------------------------------------------------------------------------
Invalid :refer, var goog.string/format does not exist
--------------------------------------------------------------------------------
2 | (:require
3 | [shadow.dom :as dom]
4 | [shadow.animate :as anim]
5 | [shadow.cljs.devtools.client.env :as env]
--------------------------------------------------------------------------------
WARN app.server: Port 8080 was not available, retrying with 8081
:point_right: App server available at
#2023-08-2312:20danbuneaI did see that in deps.edn, we have an older version:
:dev
{:extra-paths ["src-dev"] ; fix uberjar java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
:extra-deps
{...
thheller/shadow-cljs {:mvn/version "2.22.10"}}#2023-08-2312:49Geoffrey GaillardWe saw this. We suspect a corrupted .shadow-cljs cache.
We are not yet clear on the root cause.
You can try to
• stop the shadow process (not just the build, stop the JVM)
• rm -rf .shadow-cljs
• restart the shadow build
It’s worth noting we only saw this behavior with watch, never with compile or release#2023-08-2313:02danbuneaHey, thanks for answering:
• ✅ remove the cache .shadow-cljs
• ✅ stopped everything
• 🔴 clj -A:dev -X user/main#2023-08-2313:02danbuneafailure again, same error#2023-08-2313:03Geoffrey Gaillard~I see ~#2023-08-2313:12danbuneano, that's an nginx, I can try without it#2023-08-2313:12danbuneafrom something else#2023-08-2313:13Geoffrey GaillardI can reproduce the issue with a clean repo clone. Looking into it#2023-08-2313:13danbuneathank you. My use case of using electric is very much dependent on datomic so I thought I could start from this example#2023-08-2314:49Dustin GetzI do not recommend starting here, we haven't touched it in ages#2023-08-2314:50Dustin GetzI did host it this week, and am working on unifying how builds work in our dozen or so repos this week#2023-08-2314:50Dustin Getzbut your first day using electric should be the starter app#2023-08-2315:14Dustin GetzThough now that Datomic is free, I'll make the starter app use Datomic#2023-08-2315:15Dustin GetzIf you want the contributor badge lmk and you can send a PR if you like#2023-08-2315:16Geoffrey GaillardWe have a minimal repro for the shadow build issue. We’ll address it. Thank you for the report 👍#2023-08-2319:13denikrunning into the same issue#2023-08-2319:14denikas is this user https://clojurians.slack.com/archives/C6N245JGG/p1691177869837179#2023-08-2407:39henrikFWIW, I’m getting this with v2-alpha-422-g96e5c0a5 as well.#2023-08-2407:40henrikWe clear .shadow-cljs on every start, so that’s not a viable workaround, it seems.#2023-08-2412:18xificurCI pushed a new commit which fixed this on my end, can someone from this thread double check on their end? The git coords to use are
io.github.hyperfiddle/electric {:git/sha "22937f751db7258ac0be0337833c036dbc4879f2"}#2023-08-2418:05denikfixed it for me#2023-08-2419:59xificurCnew maven coordinates are out#2023-08-2314:46Garrett HopperHow feasible is it to use Malli https://github.com/metosin/malli/blob/master/docs/function-schemas.md on Electric functions?#2023-08-2315:01xificurCsince an e/fn has the same structure as a clojure fn it should be doable. I'm not sure how malli checks functions and how much effort it would be though. Are you familiar with malli's internals?#2023-08-2316:19Garrett HopperI am not unfortunately; I'm just now trying to integrate it.
I may resort to just using instrumented validation inside the functions for arguments, as many of the other feature of function schemas likely aren't applicable to Electric. (Generative testing, generation functions, etc.)#2023-08-2316:21Garrett HopperMost of the function schema/instrumentation functionality likely won't work, as Malli expects to work with plain functions which it can wrap in another fn. (Which then wouldn't have the Electric metadata)#2023-08-2316:59Garrett HopperAre Electric functions expected to support https://clojure.org/reference/special_forms#_fn_name_param_condition_map_expr?#2023-08-2317:21Garrett HopperOr varargs for that matter#2023-08-2317:22Dustin Getzvarargs are actually just released yesterday to clojars i haven’t announced it yet, see change log #2023-08-2317:22Garrett HopperOh, nice 🙂#2023-08-2317:22Dustin Getzcondition. maps- no, i’ll add a ticket. Our compat maturity is currently adhoc we implement what we need. When we come up for air (after differential electric, incremental builds, optimistic ui) we will perhaps make a compatibility matrix and ratchet it down#2023-08-2317:23Garrett Hopper👌 Makes sense; thank you 🙂#2023-08-2317:23Dustin Getzv2-422 (about to announce) has several compat impeovements #2023-08-2318:09Garrett HopperThis feels like a reasonable usage for Malli validation of function inputs
(e/defn Button [& args]
(let [schema [:catn [:x string?]]
{:keys [x]} (m/coerce schema args)]
(dom/button
(dom/text (str "Click me! " x)))))
It doesn't provide the same niceties as Malli's builtin function schemas though. (Output validation, clj-kondo integration, generators, etc.)#2023-08-2316:59Garrett HopperAre Electric functions expected to support https://clojure.org/reference/special_forms#_fn_name_param_condition_map_expr?#2023-08-2317:42Garrett Hopper@dustingetz, ~seems like something may have changed with ~
(User error)
Failed to execute 'appendChild' on 'Node': This node type does not support this method.
in ( hyperfiddle.electric-dom2/new-node #object[Text [object Text]] button ) in app/books.cljc
in case default branch
in (case 1 ...)
in reactive (defn Button nil ...) in app/books.cljc line 16
in ( hyperfiddle.electric-dom2/-googDomSetTextContentNoWarn #object[Text [object Text]] <exception> ) in app/books.cljc
in case default branch
in (case 0 ...)
in reactive (fn nil ...)
in (try ...)
#2023-08-2317:43Garrett HopperI can come up with a minimal reproduction if it's not immediately clear what might've changed about this behavior.#2023-08-2317:43Dustin Getzwe will take a look, might not be until tomorrow sorry 😕#2023-08-2317:44Dustin Getzfrom a quick glance - that's a lazy seq, does making it strict fix the issue?#2023-08-2317:45Dustin Getzi dont see how that could actually be something that changed#2023-08-2317:45Garrett HopperThe bindings? I don't think they have anything to do with it.#2023-08-2317:45Garrett HopperOk, I'll verify that it's the version change that caused it.#2023-08-2317:45Dustin GetzOh, i believe that it could be a breaking change in v2-422#2023-08-2317:48Garrett HopperHmm, this does appear to work in isolation, so it may not be the initial problem I thought it was.
(def start-reactor
(let [node (js/document.createElement "div")]
(.appendChild js/document.body node)
(e/boot
(binding [dom/node node]
(core/App.)))))#2023-08-2317:48Garrett HopperI'm not sure why the dom/node binding isn't working as expected in the full example though. :thinking_face:#2023-08-2317:51Dustin Getzis there react/reagent interop?#2023-08-2317:52Garrett HopperNo, nothing like that#2023-08-2317:53Garrett HopperAdding a breakpoint here, is it expected that a child text type would be called first before the parent button?#2023-08-2317:54Garrett Hopperparent is undefined for both calls 😕#2023-08-2317:54Dustin Getzparent should not be undefined#2023-08-2317:56Dustin Getzi dont seem to be looking at the same code as you because core/App is not defined#2023-08-2317:56Dustin Getzhttps://github.com/garrett-hopper/ElectricStorybookExamples/blob/main/src/app/core.cljc#2023-08-2317:57Garrett HopperI'm sorry. I think I was just being dumb and had a component being mounted inside dom/text. :man-facepalming:#2023-08-2317:57Dustin Getzwhere#2023-08-2317:58Garrett HopperNot in the macro I sent you, but where I'm using it, at some point I'd tried to test something and apparently ended up with
(defbook Button
(dom/text (Button. 5)))
#2023-08-2317:58Garrett HopperSo when Button when to mount, it obviously didn't have a parent since it's in a text node#2023-08-2317:58Dustin Getzand you're saying this worked in v2-349?#2023-08-2317:59Garrett HopperNo, I don't believe so. I think it was around the same time that I had upgraded that I was also doing the :pre/:post testing and instead of Button, I had an electric function that actually returned a string rather than trying to mount child components.
Entirely user error; I apologize 😕#2023-08-2318:00Dustin Getzok#2023-08-2410:23pezIf I am using e/for on the client to loop over data on the server. Then need to loop over the data again, transforming it a bit differently. Will the data be fetched from the server again, or does it depend? (Let me know if my question doesn’t make sense, and I’ll try to rephrase it.)#2023-08-2410:26xificurCI'm not sure, but e/for is differential, I wouldn't worry about it unless you see slowdowns#2023-08-2410:27xificurCjust to be sure, your e/for for server data is on the server side, correct?#2023-08-2410:27xificurC(e/server
(e/for [x server-data]
(e/client ...)))#2023-08-2410:27Dustin Getzhe has 2 loops#2023-08-2410:29pezMy e/for is on the client.#2023-08-2410:32pezSomething like
(e/server
(let [data (datomic-pull ...)]
(e/client
(e/for [x data]
...))))#2023-08-2410:36pezThis happens in a function. And calling the function with different parameters makes different things happen in the e/for body. My performance is a bit horrible and I want to get a better understanding of where to attack it.#2023-08-2410:37Dustin Getzi’ll help you in a bit, will be at my desk in 30#2023-08-2410:37Dustin Getzon mobile now#2023-08-2410:38pezNo hurries. I’ll try to put some measurements in.#2023-08-2410:38Dustin Getzdon’t bother, there are some super low hanging fruit to address#2023-08-2410:38Dustin Getzcan you post a snippet that has both loops in it?#2023-08-2410:39Dustin Getzthis explains how to reason about network transfer in e/for https://electric.hyperfiddle.net/user.demo-system-properties!SystemProperties#2023-08-2410:40pezI’ll start with reading that. 😃#2023-08-2410:41pezThere’s only one loop. The pseudo-code I posted is better to express it than the rambling prose in my OP…#2023-08-2410:43Dustin Getzalso https://clojureverse.org/t/electric-clojure-a-signals-dsl-for-fullstack-web-ui/9788/32?u=dustingetz#2023-08-2410:44Dustin Getz#2023-08-2410:45Dustin Getzthis is immediately better, because if you imagine a line from data to data it doesn't transfer, only x to x transfers. Meaning with this phrasing only x will stream over network (when changed/added/removed), not the whole collection data#2023-08-2410:45Dustin GetzThere is another aspect to consider, i may need to see your whole code#2023-08-2410:49pezI can show you the code in a DM. 😃 While you were typing all that on your phone I wrote this:
(e/defn foo [y]
(e/server
(let [data (datomic-pull ...)]
(e/client
(e/for [x data]
(dom/text (str x ":" (* x y))))))))
(foo. 1)
(foo. 2)
Taking your answer as a yes, that data will be traveling twice here?#2023-08-2411:02pezAnd now I’m having pretty decent performance. With and without your suggested change… I restarted the REPL in between.#2023-08-2411:03pezLunch break here.#2023-08-2414:18pezCan I use something like goog.functions/debounce from electric code without wrapping it in some way?#2023-08-2414:19pezLike so:
(e/defn add-pinching-listeners [element]
(e/client
(let [debounced-fn (gfn/debounce
(fn [dist]
(println "wheel" dist)
(swap! !scale-factor + (/ dist 1000)))
100)]
(.addEventListener element "wheel"
(fn [e]
(.preventDefault e)
(debounced-fn (.-deltaY e)))))))
The event handler works without debounce, but with the debounce there it is never called. And I do see this in the shadow output:
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Use of undeclared Var goog.functions/debounce
--------------------------------------------------------------------------------
#2023-08-2414:26pezI wrote my own debounce for now, but still curious. 😃#2023-08-2414:45Dustin Getzgoog imports work (all valid clojurescript is valid in electric minus any todos), there are some minor rough edges in the compiler to work out, can i see your require statement?#2023-08-2414:48pez(ns app.todo-list
(:require contrib.str
#?(:clj [datascript.core :as d]) ; database on server
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
[hyperfiddle.electric-svg :as svg]
[app.data.test :as data]
[goog.functions :as gfn]))
It seems it is this require that causes the error message. But I can evaluate gfn/debounce in the repl and it gives me a Function object.#2023-08-2414:48Dustin Getzoh#2023-08-2414:48Dustin Getzguard it in #?(:cljs ...) as it is not valid clojure#2023-08-2414:48Dustin Getzexample: https://github.com/hyperfiddle/electric/blob/22937f751db7258ac0be0337833c036dbc4879f2/src/hyperfiddle/electric_dom2.cljc#L4#2023-08-2414:48pezAh, I think I saw it when you wrote oh. 😃#2023-08-2414:48Dustin Getzi do it every time#2023-08-2414:49Dustin Getzalso this crap gets me every time: #?(:cljs (:require-macros [hyperfiddle.electric-dom2 :refer [with]])))#2023-08-2414:51pezIt’s not Electric specific. I forget to guard in cljc files all the time…#2023-08-2414:52pezBut Electric makes me forget about the boundary even more. I guess that’s just how it is. 😃#2023-08-2414:53Dustin GetzYeah same, maybe we can do something about it next year (as Electric gains momentum we'll be able to improve ClojureScript itself for example) but for now it is what it is#2023-08-2414:54pezI still get the warning, though…#2023-08-2414:54Dustin Getzit may be related to your usage of the REPL#2023-08-2414:55Dustin GetzI'll pair with you at some point, for now can you try just restarting#2023-08-2414:57Dustin Getzfwiw https://github.com/hyperfiddle/electric/blob/22937f751db7258ac0be0337833c036dbc4879f2/test/hyperfiddle/goog_calls_test.cljs that are confirmed working with a test#2023-08-2415:04pezI’ve already tried restarting the repl. 😃 Wiped the .shadow-cljs directory too, even though Thomas Heller says that is never necessary. I’m probably still holding something wrong. But we can look at it later. I’ll use my own debounce for now.#2023-08-2416:56pezIn case someone else finds this thread in the future: After an off-line chat with Dustin, I now know that using this kind of debounce is a bad idea in the context of Electric reactivity. You want to go for idiomatic FRP debounce instead.#2023-08-2417:10Dustin GetzFor the record – I am developing a missionary tutorial that starts with approximately this use case – coming at some point this fall#2023-08-2417:10Dustin Getzm/observe is in my opinion the best entry point to learning missionary, especially from an Electric perspective#2023-08-2417:30weican't wait for the tutorial. i've been slowly wrapping my head around missionary but it'd be nice to see more examples, especially in an electric context#2023-08-2417:39xificurCthe undeclared-var warnings are fixed in the latest release. There will be a quick bugfix release though (due to a problem up a few threads in this channel)#2023-08-2419:16pezI want to discuss and learn about idiomatic FRP debounce in general, and implementing pinch zoom in particular. ->🧵#2023-08-2419:19pezI’m reading Dustin’s https://www.dustingetz.com/#/page/signals%20vs%20streams%2C%20in%20terms%20of%20backpressure%20(2023) and wonder how vanilla event listeners and regular debounce fit there. Is that streams?#2023-08-2419:29pezWhen implementing pinch zoom, I want to be able to use my laptop’s trackpad. The only reliable event I’ve found for this is wheel. On Chrome (possibly in other browsers) I can recognize a pinch gesture by that the event has ctrlKey: true . Then I get events as I pinch. So far I’ve been reading the deltaY property of the event, which is a floating point number, which scale I am not quite sure how to interpret. Small pinches give things like 0.5, and I have seen double digit values. Negative values is whatever the opposite of pinch is.#2023-08-2419:32pezAs I pinch I get several events. I still don’t quite understand the significance, but it seems like the deltaY gets delivered relative to when the previous event was fired. I’ll pause here now to make a test and maybe I can describe it better after this.#2023-08-2419:42Dustin GetzWhat business domain concept is the pinch event being used to zoom? I.e. you zooming a 2D canvas or a 3D scene and what is the API to do that, is there math?#2023-08-2419:42pezThis is output from:
(.addEventListener element "wheel"
(fn [e]
(when (.-ctrlKey e)
(.preventDefault e)
(println (.now js/performance) "- wheel - deltaY:" (.-deltaY e) "deltaX:" (.-deltaX e))#2023-08-2419:46pezbusiness domain concept: I’m zooming a timeline, very much like a timeline in a video editor. The data is frames with timestamped readings from a machine, at about 10fps. Various aspects of this data is rendered in the timeline. Zooming decides how much horizontal space each frame takes, if that makes sense. Pinch in the timeline of Final Cut or Adobe Premier or similar app, and it is very similar to what I want to achieve.#2023-08-2419:46Dustin Getzwhat rendering technology#2023-08-2419:48pezI’m currently using SVG to render. But it could be canvas or WebGL or whatever, I guess.#2023-08-2419:51pezI control the zoom level with a variable frame-width (which is pixels/ms) and a scale-factor (1 = 100% zoom).#2023-08-2419:52Dustin GetzAh got it, can i see any code you have for scaling the canvas?#2023-08-2419:53pezDoes this count as that?
(e/client
(dom/div (dom/props {:id "svgContainer"})
(let [view-width (-> js/document
(.getElementById "svgContainer")
(.-offsetWidth))
svg-height (+ ruler-h trolley-pos-g-h gap)
scale-factor (e/watch !scale-factor)
frame-width (* scale-factor fpms (/ wide-screen-px wide-screen-ms))
svg-width (max view-width (* frame-width duration))]
(svg/svg (dom/props {:id "timeLineSvg"
:width svg-width
:height svg-height
:viewBox (str "0 0 " svg-width " " svg-height)})#2023-08-2420:02pezNow, say that I can’t render every event. (Because I’m doing things non-optimal ways or because it is running on a slow machine.) In that document I read this:
> If your app chokes on a burst of events, you want to skip ahead and render the final state without bothering to render all the intermediate historical states.
Which applies, but not fully with the wheel events, I think. I think I rather need to:
1. Pause trying to re-render before a render is done
2. Accumulate all pinch-deltas in the pauses and re-render using the accumulated result
(Currently re-render is done by swapping on the !scale-factor atom, which we are watching in the code I pasted above.)#2023-08-2420:04pezIn the same document I read:
> reduce is used to convert a stream into a signal, by accumulating the events into a current value.
Which seems to apply?#2023-08-2420:05pezI could have the mechanics I need wrong. But it’s how it looks like to me after a day with this. 😃#2023-08-2420:06Dustin GetzHere is how I would think about it: we want to use missionary's continuous flow operators to m/relieve backpressure at the time of the event, dumping events into a local accumulator (by summing/transforming the stream of delta-x into a "current zoom factor"), and then the UI process can lazily sample the current value of the zoom factor as fast as it can#2023-08-2420:06pezIn https://hyperfiddle.github.io/#/page/connect%20electric%20code%20to%20a%20javascript%20callback, it looks a bit like I see part of what I need to understand.#2023-08-2420:07Dustin Getzthis case is a bit more intricate, mostly because you can't click at 120fps but pinching a trackpad actually does need to update at 120fps#2023-08-2420:11pezAnyone reading the output from my pinch test above might wonder how I pinched: I started by pinching out (spreading my thumb and my finger) slowly, then paused a bit, pinched out some more, then pinched inwards, pausing a bit and pinched more inwards.#2023-08-2420:13pez> then the UI process can lazily sample the current value of the zoom factor as fast as it can
My emphasis. How do I know when I’m ready to sample the current value?#2023-08-2420:14Dustin Getzelectric entrypoint controls that#2023-08-2420:14Dustin Getzthe big idea here is that we've decoupled the event producer (the firehose of trackpad events) from the event consumer (the svg rendering/animation)#2023-08-2420:15Dustin Getzthe producer is eager (because callbacks are eager, once your callback got called you can either deal with the event or drop it, and dropping it is a correctness bug)#2023-08-2420:17Dustin Getzwhat you DON"T want is for each of those thousands of events to trigger a render, you will overwhelm the device. The trackpad might even be faster than 120fps, the hardware is capable of lets say 1000fps, i'd have to research how exactly the browser interface works#2023-08-2420:18Dustin Getzso instead of rendering every event at 1000fps, you "relieve backpressure" by directing the firehose of events into a local buffer asap. You do a little math transform (which is fast) and then buffer that state. The computer can keep up with the firehose if all it's doing is e.g. addition#2023-08-2420:19Dustin Getzm/relieve is that transition point, eager input and lazy output#2023-08-2420:19pezAh, I think a thing I did wrong in my experiments today was to reset the accumulated value every time the debouncer updated my scale-factor. I should probably just use keep accumulating? (Disregard that the debouncing was the wrong way to do it.)#2023-08-2420:20Dustin GetzI'd have to look closely to figure out what happened in your experiment, concurrency is hard and this is 100% a concurrency problem, you are now a concurrency engineer#2023-08-2420:20Dustin Getz🙂#2023-08-2420:20pezIt was bound to happen some day!#2023-08-2420:23pezI’ll have a look at m/relieve tomorrow. Sounds like fun! Now will catch some sleep and see what effect that has on how I see the problem.#2023-08-2420:23pezI hope concurrency engineers sleep well. 😃#2023-08-2504:27Stef CoetzeeMaybe just not a lot of it in one unbroken stretch! 😉#2023-08-2509:06pezNow I tried to defer all this by shifting my focus to another part of my experiment. Only to (I think) run into the same problem category. 😃 Now it’s scroll events. I need to deal with when the timeline is scrolled and compute a new clipped view/sliding window. Scroll events also can fire very rapidly. Though I could be thinking wrongly about it. I remember seeing an example with a sliding window on vertical scroll in the presentation to London Clojurians, that I think is very similar to what I want to do. I’ll have a look on the example and loop back here.#2023-08-2509:17xificurCthe https://github.com/hyperfiddle/electric/blob/master/src-docs/user/demo_explorer.cljc uses the unfinished https://github.com/hyperfiddle/electric/blob/master/src/contrib/gridsheet.cljc component. It https://github.com/hyperfiddle/electric/blob/7fb4aa8198a26c1e5dd5221e522ced053bb28c45/src/contrib/gridsheet.cljc#L36 and then does some math+css to render a grid#2023-08-2509:33pezThanks! That https://github.com/hyperfiddle/electric/blob/7fb4aa8198a26c1e5dd5221e522ced053bb28c45/src/hyperfiddle/electric_ui4.cljc#L322 thing (what is a term for it?) looks much saner than what I was going with. I even see m/relieve in there. 😃 However, it only deals with vertical scrolling, so I might use it to roll my own for now.#2023-08-2509:40xificurCthe < suffix on a function denotes the function returns a continuous flow. So a proper term would be what, scroll state continuous flow constructor? 🙂#2023-08-2509:41xificurC> it only deals with vertical scrolling
it listens to scroll events and samples the element's scrollTop, scrollHeight and clientHeight. Adding the width should be simple enough#2023-08-2516:59pezI forgot to report back. It worked great for my use case. I don’t quite understand why it doesn’t lag, but it makes me happy that it doesn’t.#2023-08-2517:58xificurCgreat! Re pinch, most likely yes. Just use + as the m/relieve reducing function, to collapse the deltas. On scrolling the .-scrollHeight etc reads are always the latest value, in the pinch event it seems you're getting deltas, so you have to accumulate the values#2023-08-2519:25pezWith @U02KSQWSHSQ’s (my client) permission I can share a video of the timeline as I am zooming (with buttons for now) and scrolling. I’m thinking it works better as a description of the problem than my fumbling prose above. What happens at both scroll and zoom is that a sliding window over the timeline data is recalculated and this is then looped over with e/for rendering the green graph. The graph might look like we are drawing polygons, but we are actually drawing a lot of thin bars. For now we draw all frames, at a frame-rate of 10 fps. The numbers in the ruler is seconds. Note: While it still takes a while to render all bars, at no point the scrolling lags. This is the part I am happy about, but don’t quite understand.
I will probably open a new discussion about strategies for speeding the actual rendering up. Ben and I have some ideas, and we just need to experiment a bit before we can ask for help in any intelligent way. But first I’ll take a stab at the pinch zooming again. Wish me luck!#2023-08-2519:27Dustin GetzThe best way to speed up hardcore rendering may be non-obvious, we need to be involved in that#2023-08-2519:27Dustin GetzAs a baseline, the datomic browser demo is our current perf demo - https://electric-datomic-viewer.fly.dev/(:app.datomic-browser!attribute,:abstractRelease!artistCredit)#2023-08-2519:28Dustin Getzstreaming 10k rows through the dom as you drag the scrollbar#2023-08-2519:29pezI’m dragging like crazy here. So smooth and nice!#2023-08-2519:32Dustin GetzDo you have a specific question? The question is why is this way fast and why is what you did before slow?#2023-08-2519:49pezI think my question is similar to that. The behaviour I had with my pinching implementation was that the the renderings seems to be batched and when they couldn’t keep up with my zooming speed it got clunky and it looked like it did all the renderings in between the different states. Dunno if you can follow that description, but compared to this scrolling the difference is that the scrolling follows exactly my command, and the renderings are not completed in some old state. I’d say the scrolling and the rendering behaviour is as smooth and good as it can get, given the inefficiencies in the actual rendering. I think I understand why my old solution is laggy in the way it was, but I don’t understand what mechanisms in the new solution that makes the rendering stop and restart from the most recent state (or even if that is what happens). Including the code for the continuous flow of scroll events, for reference.#2023-08-2519:50Dustin GetzThere are a couple ways it could have happened, I would need to look closely at the old code#2023-08-2519:50Dustin GetzI have a good guess though#2023-08-2519:50pezA conjecture even? 😃#2023-08-2519:51Dustin GetzElectric doesn't automatically relieve backpressure, the current reactive rendering strategy is to render as fast as possible, as soon as anything changes. (Compare to another strategy we could choose, which is to render at up to the browser animation rate with requestAnimationFrame)#2023-08-2519:52Dustin GetzThe result of that is that Electric re-rendered with each individual touchpad event – my blind guess#2023-08-2519:52pezThat was my guess too, and why I reached for debounce.#2023-08-2519:52Dustin Getzright#2023-08-2519:55Dustin Getzthe rule of thumb to remember is: any time you have a callback/event from a user interaction, you want to relieve it asap. "Don't backpressure the user"#2023-08-2519:55Dustin GetzThe user wants to type fast (or scroll or whatever) and never ever wants to be prevented from typing fast just because the rendering can't quite keep up#2023-08-2519:55Dustin Getzso m/relieve is how you decouple the user interaction from the rendering#2023-08-2519:57Dustin GetzOn the other hand, you don't want to just blindly relieve everywhere, because decoupling the producer from the consumer has a consequence: it can and will skip frames#2023-08-2519:58Dustin Getzif your effects are (dom/text x) then you don't actually care if you skip a dom write every now and then, because you only ever want to see latest#2023-08-2519:59Dustin Getzbut if your effect is (println x) or (submit-database-transaction x) then you most definitely, absolutely care that every single write occurs and is never skipped#2023-08-2520:00Dustin Getzso m/relieve is something you only want sometimes, and that decision is essential complexity that the programmer must manage on a case by case basis#2023-08-2520:01Dustin GetzThis is why Missionary feels complicated, there are a number of primitives that simply can't be automatically done for you, Missionary's philosophy is to have each primitive do one and only one thing and assume the programmer knows best#2023-08-2520:04bdbrodieWorth building a correct intuition for each primitive based on source code, rather than treating them like magic black boxes?#2023-08-2520:04Dustin Getzi dont understand the q#2023-08-2520:05Dustin GetzI can say that Missionary's docs and docstrings need to be rewritten (are being rewritten)#2023-08-2520:05Dustin Getzwe will teach you#2023-08-2611:20pezSo, now I have a continues flow of accumulated pinch state.
#?(:cljs (defn pinch-state> [pinchable]
(m/observe
(fn [!]
(let [sample (fn [e] (! (when (.-ctrlKey e)
(.preventDefault e)
(.-deltaY e))))]
(.addEventListener pinchable "wheel" sample #js {"passive" false})
#(.removeEventListener pinchable "wheel" sample))))))
#?(:cljs (defn pinch-state< [pinchable]
(->> (pinch-state> pinchable)
(e/throttle 16) ; RAF interval
(m/reductions {} 0) ; What's the significance of passing empty map as `rf` here?
(m/relieve +)
(m/latest identity))))
The flow as such seems to work, but then I don’t really know what to do with it. Currently I have an atom with the zoom-factor and if I swap on it, with a function using the pinch values as input, the behaviour is somewhat correct. (See attached video). I don’t think this is the right direction. Sometime I get significant slowdowns. I think maybe I should create a continues flow of zoom state. I don’t quite understand yet how to do that. The zoom state is a function of the pinch delta… But the zoom factor can be altered from other sources, like the buttons in this example.
Also, the zoom level invalidates the scroll position, which as some minimum of sanity should retain its relative position on the timeline. Thus, the flow of zoom state probably needs to have some of the previously/currently (I’m unsure which is the correct model of it) rendered state and update the scroll when the new graph width has been rendered. This flow can’t be risking to lose track of the previous scroll position before updating with the new one. Right now I have no idea how to achieve this.#2023-08-2510:16mmerHi I know that this channel has exchanges about people who have got things running. However, when I look at the example projects I am confused by what I see. I got the impression that electric clojure meant writing the application in one spae and the compiler generated the cljs and the clj for you. But when I look at the example projects I find both cljs and clj files. I was expecting to have a single file src (or at least a single entry point) for the project. Sorry if this is a really dumb question.#2023-08-2510:29Dustin Getzmainly because clojure auto-loads user.clj but NOT user.cljc#2023-08-2510:30mmerJust so I understand - the user files are part of the scaffolding for the project and the todo_list.clj is the electric source?#2023-08-2510:31Dustin Getztodo-list is cljc#2023-08-2510:33Dustin Getzbut yes, we have to layer on top of the existing clojure/script infrastructure like shadow, http servers, java .jars etc. So from the platform perspective there are two entrypoints one for each actual platform process (Browser and JVM)#2023-08-2510:33Dustin Getzthe Electric compiler is a macro, it compiles during macroexpansion#2023-08-2510:34Dustin Getzty for the question i will take a fresh look and see if we can remove some boilerplate now that things are more mature#2023-08-2514:18Dustin GetzElectric v2-alpha-428 is released, see https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md.
• Greatly improved Clojure/Script compat (varargs, apply, multi arity, #js, java & js interop forms e.g. js/alert) – please report bugs and capability gaps
• Fixed many ClojureScript compiler warnings, especially wrt type hints e.g. ^js. Please send us any warnings you encounter so we can iron out remaining edge cases
• core.match and clojure.logging now work seamlessly in Electric
• https://github.com/hyperfiddle/fulcro-electric-template and https://github.com/hyperfiddle/fulcro-electric-template/blob/master/src/main/app/server_components/http_server.clj adapter examples
• bugfixes
Please upgrade and click the check after it worked#2023-08-2518:33Joseph WinstonI'm trying to update to Electric v2-alpha-428 and I am seeing the following error#2023-08-2518:34Joseph WinstonEncountered error when macroexpanding hyperfiddle.electric/boot.
Unable to resolve symbol: e/*http-request*
{:file "app/dashboard.cljc", :line 73, :column 29, :end-line 73, :end-column 45, :in [e/*http-request*]}
My use is exactly the same as in
src-docs/user/demo_chat_extended.cljc: (let [session-id (e/server (get-in e/*http-request* [:headers "sec-websocket-key"]))"
Suggestions?#2023-08-2518:40Dustin Getzhmm our examples app is 4 commits behind the release, i will upgrade it to be certain#2023-08-2518:41Joseph WinstonThanks for taking a look:+1:#2023-08-2518:48Dustin Getzthe example apps work on my machine, CI is ~deploying~ works, no error#2023-08-2518:48Dustin Getzcan you add me to your repo?#2023-08-2519:58Joseph WinstonThanks for checking. I'll keep digging since I can't add you to the repo.#2023-08-2600:18weiNo errors, although I did get a few "Cannot infer target type in expression" warnings that I didn't see before.
------ WARNING #1 - :infer-warning ---------------------------------------------
File: ./user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
Cannot infer target type in expression (. G__53289 sendPayment G__53290)
--------------------------------------------------------------------------------
9 | (binding [hyperfiddle.electric-dom2/node js/document.body]
10 | (app.main/Main.))))
11 |
12 | (defonce reactor nil)
--------------------------------------------------------------------------------
...
#2023-08-2601:04Dustin Getzsend us the exprs and the warning, we are trying to iron this out#2023-08-2607:37xificurCSometimes these go away after rm -rf .shadow-cljs#2023-08-2614:57Dustin Getzrm -rf .shadow-cljs indeed fixed these warnings on my machine. I believe this is NOT a shadow-cljs issue, this is Electric/Shadow hacks that muck with the shadow build state#2023-08-2614:57Dustin GetzIIUC, only when upgrading Electric versions could this matter, in the event that we have changed the electric<>shadowcljs hacks#2023-08-2615:01Dustin GetzI've added this instruction to the changelog#2023-08-2615:19Dustin Getzhmm, the warnings came back for me#2023-08-2706:29weibtw this is the snippet that produces the warning (I had sent this to Dustin but not the channel)
(e/defn Deposit-UI [{pubkey :xt/id :as user}]
(let [!deposit-amount (atom 42)
deposit-amount (e/watch !deposit-amount)]
(dom/div
(ui/long deposit-amount (e/fn [v] (reset! !deposit-amount v)))
(ui/button
(e/fn []
(let [{:invoice/keys [payment-request]}
(e/server
(splitter/create-invoice! (e/snapshot pubkey) deposit-amount "deposit"))]
(.sendPayment js/webln payment-request)))
(dom/text "deposit")))))#2023-08-2603:20Garrett Hopper@dustingetz, looks like varargs with a :or {} default map may not be supported yet?
(e/defn Example [& {:keys [x]
:or {x 1}}]
(dom/div
(dom/text "X: " x)))
(e/run
(Example. :x 2))
(Shows 1 instead of 2)
Also calling Example with no args results in a You called Example with 0 arguments but it only supports 1#2023-08-2603:21Garrett HopperErr, I guess the :or is working, but it's always being used rather than the input value.#2023-08-2603:23Garrett HopperUh, if I do :as x in the varargs map, it shows that it's a vector rather than a map#2023-08-2603:30Garrett Hopper(e/defn Example [& xs]
(let [{:keys [x] :or {x 1}} (apply hash-map xs)]
(dom/div
(dom/text "X: " x))))#2023-08-2604:31Garrett HopperAlso potentially a feature request to support apply on Electric functions?
(apply Example. [:x 1])
Assuming it makes sense to do so :thinking_face:#2023-08-2607:39xificurCThanks, I'll take a look at these on Monday, I might have missed som edge cases#2023-08-2607:39xificurCThere's e/apply#2023-08-2613:38Dustin Getzthank you for the reports#2023-08-2807:57henrik@U09FL65DK Pathom has implemented args destructuring for defresolver, perhaps you could “borrow” the code for it.#2023-08-2809:11xificurC@U5JUDH2UE I didn't realize map varargs requires special casing and forgot to test it, thanks for the report! I landed a new commit supporting this up to https://clojure.org/news/2021/03/18/apis-serving-people-and-programs
I couldn't reproduce the no-args-throws case with your above snippet though#2023-08-2617:41braai engineerI'm building an online booking / quoting app for my sound rental business using Electric. I understand that static-site rendering is not a target for Electric, but is there a way to get a once-off HTML dump (at e.g. build time) of an Electric component so I can populate the mount DIV with some text to make Google happy? This is not for client-side hydration - just a once-off to get HTML out for basic SEO. Sure I could copypasta the output from console but I keep tweaking stuff in Electric and would like to automate it.#2023-08-2618:18Dustin Getzyou'll need to code it#2023-08-2618:19Dustin Getzi highly highly highly highly recommend next.js/react.js for this kind of app, the toolchain is quite excellent#2023-08-2621:19Vincenttonsky's rum also has "serverside rendering" support for components, if your components are not complicated maybe having duplicates in rum for sake of google SEO is a viable option. an idea.#2023-08-2811:48asolovyovhi! I'm trying to get my feet wet with Electric, and got (httpkit) adapter listening, it reports "Connected." into a browser console, reload with shadow-cljs is working also. Even DOM nodes are created, but they are empty, no content in there - and it's just a static string for now. I took most of my setup from electric-starter-app , and my main component is (e/defn Main [] (e/client (dom/h1 "Test"))) .
Not sure where to look at to get it working 🙂#2023-08-2811:52Dustin GetzI recommend starting from a fresh clone of the starter app#2023-08-2811:52Dustin GetzIs there a reason for starting with a custom configuration?#2023-08-2811:53Dustin Getzah the issue is missing dom/text – (dom/h1 (dom/text "Test"))#2023-08-2811:53asolovyovahhh thanks 😁#2023-08-2811:53asolovyov> Is there a reason for starting with a custom configuration?
I find that getting through errors helps me understand the setup better 😁#2023-08-2811:54asolovyovfantastic, thanks! 🙂#2023-08-2811:55asolovyovI've got too used to usual hiccup-like syntaxes 🙂#2023-08-2812:26danbuneaHi, I am looking into electric-starter-app, and I wonder why each todo is read again in the Todo component, since it's available in the list and could be just passed to the component, simplifying the Todo component:
(e/defn Todo-list []
(e/server
(binding [db (e/watch !conn)]
(e/client
(dom/link (dom/props {:rel :stylesheet :href "/todo-list.css"}))
(dom/h1 (dom/text "minimal todo list"))
(dom/p (dom/text "it's multiplayer, try two tabs"))
(dom/div (dom/props {:class "todo-list"})
(TodoCreate.)
(dom/div {:class "todo-items"}
(e/server
(e/for-by :db/id [todo (todo-records db)]
(TodoItem. todo))))
(dom/p (dom/props {:class "counter"})
(dom/span (dom/props {:class "count"}) (dom/text (e/server (todo-count db))))
(dom/text " items left")))))))
and
(e/defn TodoItem [todo]
(e/client
(dom/div
(ui/checkbox
(case (:task/status todo) :active false, :done true)
(e/fn [v]
(e/server
(d/transact! !conn [{:db/id (:todo/id todo)
:task/status (if v :done :active)}])
nil))
(dom/props {:id (:todo/id todo)}))
(dom/label (dom/props {:for (:todo/id todo)})
(dom/text (:task/description todo))))))
now there's only a e/client and only has a e/fn that contains an e/server call.
Is there a reason why the component was rereading from the server:
(e/server
(let [e (d/entity db id)
and also doing this call:
(dom/text (e/server (:task/description e))
The initial component:
(e/defn TodoItem [id]
(e/server
(let [e (d/entity db id)
status (:task/status e)]
(e/client
(dom/div
(ui/checkbox
(case status :active false, :done true)
(e/fn [v]
(e/server
(d/transact! !conn [{:db/id id
:task/status (if v :done :active)}])
nil))
(dom/props {:id id}))
(dom/label (dom/props {:for id}) (dom/text (e/server (:task/description e)))))))))#2023-08-2812:31Dustin Getzlooks like an accident, we have several copies of this app in various repos and they can get out of sync#2023-08-2812:32Dustin Getzcode it how you like but note that the number of e/client and e/server calls is not something that you should be optimizing#2023-08-2812:35Dustin Getzwe also like the entity API in concept for more fine-grained queries, though in practice there's lots more work to be done to bolt incremental view maintenance on top of a batch database#2023-08-2818:02braai engineerTrying to get a cookie store going for longer-lived sessions. Am I doing this right?
(ring/run-jetty (http-middleware resources-path manifest-path)
(merge {:port port
:join? false
:configurator add-gzip-handler
:websockets {"/" (fn [ring-req]
(adapter/electric-ws-adapter
(partial adapter/electric-ws-message-handler
(-> ring-req
(ring-session/session-request
{:store cookie-storage
:cookie-name "ring-session"})))))}}
config))
where cookie-storage is (cookie-store {:key cookie-store-key-16-bytes})
Under /auth route I am trying to set "something" in :session:
(-> (res/status response 302) ; redirect
(assoc :session "something")
(res/header "Location" (get-in ring-req [:headers "referer"])))
However, I see that :session is nil on request:
(e/server (:session e/*http-request*))
Haven't used Ring session stuff in ages. This is not Electric-specific, but not sure if I can do this with the Electric websocket handler.#2023-08-2818:06braai engineerI see this in Tesserae, but wrap-user-session is not currently used: https://github.com/lumberdev/tesserae/blob/04d168ddaf7d5cc8ee56201038a34a48df11d4d3/src/tesserae/serve.clj#L52C1-L64C1#2023-08-2818:11braai engineerHmm I added wrap-user-session to HTTP middleware and now I can see the :session on the request when it comes in, but it's not on e/*http-request* :
(defn wrap-user-session [handler]
;
(ring-session/wrap-session
handler
{:store
(cookie-store {:key cookie-store-key-16-bytes})
;
:cookie-name "quey-cookie"
:cookie-attrs {:max-age one-year-seconds}}))#2023-08-2819:26braai engineer@U050CJFRU you might know?#2023-08-2819:48telekidThese are all standard clojure / ring questions, you’ll find a much larger audience in #C03S1KBA2 #2023-08-2821:05nakkaya@U051SPP9Z See [1] its a strip down version of how set things up for authentication.
[1] https://gist.github.com/nakkaya/9087057f3c3ffc18f6fadd7a53e366ff#2023-08-2821:49Vincentgood luck i hope you figure it out.
how can it not be in e/http-request?
that confuses me
makes me think server not restarted or something
there is a working session example in one of the examples, i built from that 1#2023-09-0621:02denik@U051SPP9Z did this work for you? I’ve had some cookie issues. wondering now if that might have to do with the ws-handler not using sessions (they’re used for rest)#2023-08-2818:59braai engineerWhy does (e/server (log/debug "Electric request:" (pr-str e/*http-request*))) print out nil for the HTTP request if I put it in root component?#2023-08-2819:44Dustin GetzI don't understand, can you link me to the exact line in this repo where you'd observe a nil *http-request*? https://github.com/hyperfiddle/electric-examples-app/blob/main/src/user/demo_chat_extended.cljc#L42#2023-08-2819:45Dustin Getzentrypoint is https://github.com/hyperfiddle/electric-examples-app/blob/main/src/user.cljs#L6; user-main/Main is https://github.com/hyperfiddle/electric-examples-app/blob/8d4fe58b46856ea879d0b8af15e9a1c72604d1b9/src/user_main.cljc#L272#2023-08-2819:45Dustin Getzyou can clone that repo btw, it has the working configuration for the tutorial example#2023-08-2913:10pezCan I add a <style> element to the dom somehow?#2023-08-2913:11Geoffrey Gaillard(dom/element "style" (dom/text ".class{…}"))#2023-08-2913:14Dustin Getzalso (dom/style {:border "2px solid red" :border-radius "0.2rem"})#2023-08-2913:14Dustin GetzOops, I mean yes dom/style works but it doesn't answer the q, sorry#2023-08-2913:15pezNo worries. 😃 @U2DART3HA answered the q!#2023-08-2913:15Dustin GetzNow I see there is a naming collision between style-the-element and style-the-attribute, I didn't know about that, will ticket#2023-08-2914:58KeinHi, in order to emit event in reagent block and still maintain an electric context, I store the event in an atom and e/watch it. Whenever the event-status is set, the event is dispatched and executed.
The dispatch function receives 1 argument, but cljs keeps trying to invoke the argument as function. And I’m not sure how to inspect what’s going on during executing this function.
I guess the problem relates with hyperfiddle, CLJS, and CLJS debugging methods.#2023-08-2914:59Keinissue in CLJS: https://clojurians.slack.com/archives/C03S1L9DN/p1693320510319619#2023-08-2915:00KeinBasically, I have an event defined as:
(def event-default {:status false :event {:type nil :edata nil}})
(defonce !event (atom event-default))
I set the event type and event data when I emit the function. And I use the following event system to dispatch
(e/defn event-system []
(let [event (e/watch !event)]
(when (= (:status event) true)
(dispatch-event (:event event))
(reset! !event event-default)
)))
#2023-08-2915:00KeinThe bug goes like this:#2023-08-2915:03Dustin Getzshow me dispatch-event#2023-08-2915:05KeinI’d really like to know some of the following:
[1] Why the js runtime try to call the map even I did not invoke it.
[2] What is the best practice to inspect the stack trace? I did not find any functions that I defined. There are some electric functions in the stack, but as we know do not show its passed value.
[3] In hyperfiddle, if we pass a function as arg into another wrapping function, we seem to lose the ability to see it in the stack trace. Is this an issue? How might we tackle?#2023-08-2915:06KeinDispatch event:
(e/defn dispatch-event [event]
(let [event-map {
:create create-data!
}
]
(((:type event) event-map) (:edata event))
)
)#2023-08-2915:06Dustin Getze/defn must be called with new#2023-08-2915:06Dustin Getz(dispatch-event. ...)#2023-08-2915:06KeinOhh#2023-08-2915:06Dustin Getzbtw, please capitalize Electric fns#2023-08-2915:07Dustin Getza convention which helps prevent this mistake#2023-08-2915:07KeinI forgot this one. Thanks.#2023-08-2915:07KeinDo you have some ideas about the question [2] and [3] ?#2023-08-2915:08Dustin GetzRe. debugging – electric's runtime traces are pretty good, but the compile time traces are not good#2023-08-2915:09Dustin Getzordinary Clojure macros have the same problem, if you input malformed code into a macro, the macroexpansion only makes it worse and adds even more distance between userland code and actual code that runs#2023-08-2915:09Dustin Getzour current thinking is to add a bunch of specs all over to try to detect syntax errors and such early, like Clojure 1.10 tries to do#2023-08-2915:09Dustin Getzwon't happen in 2023 unfortunately, priority is the production use cases and finishing the language semantics#2023-08-2915:11KeinSo it seems that the upper part of the error in the image above is generated via electric runtime trace? I.e.
in dynamically bound ( app.home/dispatch-event {:status true, :event #} )
in case default branch
in reactive (defn event-system [] ...) in app/home.cljc line 181
in reactive (defn Home [] ...) in app/home.cljc line 197
in (try ...) #2023-08-2915:11Dustin Getzoh you're right, my mistake#2023-08-2915:11KeinOr other places that we can exploit electric’s runtime traces?#2023-08-2915:12Dustin GetzElectric crashed on the right line here, which is enough most of the time#2023-08-2915:13Dustin Getzthe root cause of the crash is a syntax error though, malformed syntax was fed into the macroexpansion and the macroexpansion doesn't validate much today#2023-08-2915:13KeinWhat do you mean by ‘the right line’ ?#2023-08-2915:14Dustin Getzthe top line of the trace references dispatch-event, which is the line with the syntax error#2023-08-2915:14Dustin Getzif we pass a function as arg into another wrapping function, we seem to lose the ability to see it in the stack trace can you show me an example of this?#2023-08-2915:16Kein> the top line of the trace references dispatch-event, which is the line with the syntax error
I see. So now we can see the exact line but without the cause.#2023-08-2915:21Kein> can you show me an example of this?
If we have a function A [x y z] , and things go wrong with A , we can see the name A in the stack trace but without the value of x, y, and z.
However, if we instead use something like (apply A x y z) , when things go wrong with A, we even lose the information of what A is. This is the pattern I was about to mention.#2023-08-2915:23KeinI’m searching for and asking (in #CLJS) about practical ways to get the actual value in a stack trace#2023-08-2915:23Dustin Getzdo you mean clojure.core/apply or electric/apply#2023-08-2915:23Dustin GetzI think i need an example to understand#2023-08-2915:30Dustin Getz#2023-08-2915:31KeinClojure apply.
(defn A [x y z] (+ x y z)) ;will cause error
(apply A x y z)
I noticed that the apply function will invoke A regularly. So the question[3] in general might not exist.#2023-08-2915:31Dustin Getz#2023-08-2915:32KeinBut I think that the macro-expansion function in this case do cause this issue. We do not know what syntax that is passed in gets wrong.#2023-08-2915:14Jim DueyHowdy folks. Kicking the tires on Electric#2023-08-2915:51Garrett HopperIs there any clean way to e.preventDefault when using a ui4/button as a submit button in a form? (Or should I just use a button an dom/on?)#2023-08-2916:29Dustin Getzhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1678963314778609#2023-08-2916:30Dustin GetzCan you use <button role="button">…</button> ?#2023-08-2916:35Garrett HopperI wouldn't get "enter on last input to submit" behavior by default then#2023-08-2916:36Dustin Getzah, our design wants you to use a lower dom/button for HTML semantics#2023-08-2916:38Dustin Getzthe ui5 widgets are intended to have electric-native sync state semantics, user inputs will be streamed to the server as you type by default (with a ~middleware~ combinator to intercept this signal to add commit/discard semantics at various levels - field level, form level, page level etc)#2023-08-2916:38Dustin Getzui4 is just historical experiments headed in that direction#2023-08-2916:38Dustin Getz"google sheet" semantics by default if you will; which also matches PKM semantics#2023-08-2916:42Garrett HopperMakes sense; I know ui5+ stuff is still in flux, so I won't worry about falling back to dom/on workaround for now.
Form with default of preventing submit is probably one of those things that will be likely desired in that higher level eventually.#2023-08-2916:43Dustin GetzI'm a bit confused actually, are you looking to submit form state via http post (out of band of electric)?#2023-08-2916:44Garrett HopperNope, just looking to have the default browser feature of submitting a form by pressing enter. (Specifically preventing the http post)#2023-08-2916:45Dustin Getz"submit form by pressing enter" meaning trigger http action?#2023-08-2916:46Dustin Getzif you're preventing the http post then what is the browser submit event getting you#2023-08-2916:53Garrett HopperMeaning enter key triggers the button click event; even if the actual post is prevented. It's just for the accessibility of it. It could also be done manually of course, but I'd rather not try to replicate what the browser already does by default.#2023-08-2916:54Dustin Getzohh#2023-08-3019:30grounded_sageI'm looking at making Datahike work easily with Electric like the Datalevin/Datascript starters. Though e/watch does not appear to trigger the updates. When I do add-watch on the database connection it triggers the callback. I am unsure what is different with e/watch and how to debug this.#2023-08-3019:31Dustin Getze/watch is implemented in terms of the atom subscription interfaces provided by clojure.core, likely Datahike does not implement them#2023-08-3019:31Dustin Getzyou can implement the reactive subscription with m/observe, search in #CL85MBPEF for discussions related to this#2023-08-3019:32grounded_sageYea I have taken a good look at missionary today. But I still wanted to figure out why the simple solution doesn't work.#2023-08-3019:33Dustin GetzOk I now see you tried add-watch against datahike and it works#2023-08-3019:33grounded_sageThere is (:wrapped-atom conn) which returns an atom which works with the normal watcher. There is also a branch about to be merged that makes it possible to do without this.#2023-08-3019:34Dustin Getzif add-watch is working, i expect e/watch is working as well, try to confirm it with just a println#2023-08-3019:35Dustin Getznext likely issue is that equality is implemented wrong and/or hacked on whatever object is returned by the subscription callback#2023-08-3019:35grounded_sage(binding [db (e/watch conn)]
(println "This is the db: " db)
(println "the db query: " (dh/q '[:find [?e ...] :in $ ?status
:where [?e :task/status ?status]] db :active))...)
This works when I refresh the page. I can add a task with the UI.#2023-08-3019:35grounded_sageBut it doesn't trigger when I transact. I have to do the page refresh to get it printing. So I'm missing something.#2023-08-3019:36Dustin GetzI don't understand#2023-08-3019:36Dustin Getz"works" and "doesn't work" is also underspecified, please state expectations and actual#2023-08-3019:36grounded_sageAll good. I will look at what you mentioned just above.#2023-08-3020:15grounded_sageDoes seem to be an equality issue.#2023-08-3020:16Dustin Getzhttps://ask.datomic.com/index.php/859/equality-on-d-entity-ignores-db#2023-08-3116:33Vincentfascinating#2023-08-3117:36grounded_sageCurrent status:
Can transact at the repl and get the result rendered in electric. This was not working before as there was an issue with equality.
However transact wrapped in electric causes a failure. We are unsure why this is happening. It fails silently. #2023-08-3117:37grounded_sageIt spat out this at one point long after it stopped working.
ERROR:
java.lang.UnsupportedOperationException
at java.base/java.lang.Thread.stop(Thread.java:1705)
at nrepl.middleware.session$interrupt_stop$fn__22452.invoke(session.clj:198)
at clojure.core$binding_conveyor_fn$fn__5823.invoke(core.clj:2047)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1623)
#2023-08-3117:57VincentHmm binding conveyor is new to me, although it makes sense there would be some sort of binding sequence /conveyor belt... UnsupportedOperation shows up when ...
i'm thinking there might be some gap in datahike - a vague hunch.#2023-08-3117:58Dustin Getzi don’t recognize that, but try wrapping in e/offload#2023-08-3118:23grounded_sagee/offload doesn't appear to do it.
Extra information:
• when I transact inside electric the transaction works. Querying at the repl I can verify this.
• the repl eventually dies though
@U055PQH9R4M the error shows up on browser refresh#2023-08-3118:58grounded_sageI've invited you to repo @U09K620SG also happy to invite anyone else that can help surface the error.#2023-08-3119:06xificurCif the API is the same as datascript you can try to swap out the requires and see if it works. We know datascript works so it would be a user error#2023-08-3119:19grounded_sageThere is a lot happening with Datahike and it has heavily diverged from Datascript. We are working to get it functional with Electric. The silent failure makes it a little difficult to tell where the error is. #2023-08-3119:22grounded_sageI also just tested transacting from within electric with e/watch commented out. Then queried in repl. This works repeatedly.
So it is somewhere between transact inside of electric and the watch that an error is occurring.#2023-08-3119:32grounded_sageHaving a Datahike starter is interesting IMO because you can use the JDBC interface and point to a service like Supabase for a PostgreSQL backend or use the S3 backend. Easy persistence.#2023-08-3119:45grounded_sageI can actually just make this repo public since it is going to be anyways. https://github.com/groundedsage/electric-starter-datahike#2023-08-3120:15xificurCmy suggestion to put datascript in was a temporary test to rule out user errors. Having a working datahike starter app is a great and much appreciated effort!#2023-08-3120:16xificurCthe link is a 404#2023-08-3120:19grounded_sageAh it wasn't properly public. It is now.#2023-08-3120:20grounded_sageI copied the Datalevin starter someone had created. Datalevin is still in it commented out so I can switch between it for ruling out errors in userland.#2023-08-3120:32xificurCI'm not familiar with datahike's internals but from a quick glance the :wrapped-atom is some sort of internal thing? From the public API I'd use listen to receive alerts about new transactions and deref the connection at that point#2023-08-3120:35xificurC(dh/listen conn #(reset! !db (:db-after %)))#2023-08-3120:40grounded_sageWas trying to get (binding [db (e/watch conn)] ...) working out of the box. Because conn can be watched.#2023-08-3120:49whiloi will take a look into this later tonight, having electric support would be super nice#2023-08-3120:50whiloi currently work on the connection management in datahike and added the watcher lately. maybe equality has some problem, but it was always tested since we forked from datascript.#2023-09-0110:24Dustin Getzyou copied a third party starter app repo? don’t do that, you must start from ours#2023-09-0110:24Dustin Getzthird party repos are not maintained and likely junk#2023-09-0110:25Dustin Getz@U1C36HC6N datascript db equality was broken we had to fix it#2023-09-0117:29whiloi see, is your fix upstream?#2023-09-0117:29whilowe have rolling hashes already, which make equality tests faster, but need to check how far i can piggy back on them#2023-09-0117:30Dustin Getzour datascript PR is linked in the ask datomic url above#2023-09-0117:30whilothx!#2023-09-0402:39whiloi implemented a fix that does not require the identical? hack of datascript, but checks hash equality instead#2023-09-0422:26grounded_sageTested this today and it worked.#2023-09-1108:46grounded_sageI get this error when building to a jar. Has anyone seen this before?
WARN org.eclipse.jetty.websocket.common.WebSocketSession: Exception while notifying onError
java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_error__44172.invoke(electric_jetty_adapter.clj:71)
at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__44027.invoke(websocket.clj:153)
at ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6.onWebSocketError(Unknown Source)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onError(JettyListenerEventDriver.java:169)
at org.eclipse.jetty.websocket.common.WebSocketSession.callApplicationOnError(WebSocketSession.java:416)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.close(AbstractWebSocketConnection.java:209)
at org.eclipse.jetty.websocket.common.WebSocketSession.close(WebSocketSession.java:130)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.openSession(AbstractEventDriver.java:221)
at org.eclipse.jetty.websocket.common.WebSocketSession.open(WebSocketSession.java:493)
at org.eclipse.jetty.websocket.common.WebSocketSession.onOpened(WebSocketSession.java:459)
at org.eclipse.jetty.io.AbstractConnection.onOpened(AbstractConnection.java:213)
at org.eclipse.jetty.io.AbstractConnection.onOpen(AbstractConnection.java:205)
at org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:444)
at org.eclipse.jetty.server.HttpConnection.onCompleted(HttpConnection.java:401)
at org.eclipse.jetty.server.HttpChannel.onCompleted(HttpChannel.java:820)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:368)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:279)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
at java.base/java.lang.Thread.run(Thread.java:1623)
Caused by: java.lang.ClassNotFoundException: clojure.tools.logging.impl.LoggerFactory
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 27 common frames omitted
WARN org.eclipse.jetty.websocket.common.WebSocketSession: Exception while notifying onClose
java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_close__44165.invoke(electric_jetty_adapter.clj:68)
at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__44031.invoke(websocket.clj:159)
at
#2023-09-1108:50xificurCdo you have tools.logging on the classpath?#2023-09-1108:51grounded_sageHow do I check that?#2023-09-1108:51xificurCwhat command are you running to build?#2023-09-1108:51grounded_sageHYPERFIDDLE_ELECTRIC_APP_VERSION=`git describe --tags --long --always --dirty`
clojure -X:build uberjar :jar-name "app.jar" :version '"'$HYPERFIDDLE_ELECTRIC_APP_VERSION'"'
java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -jar app.jar
#2023-09-1108:52grounded_sageI am using the official electric-starter-app and swapped out Datascript for Datahike.#2023-09-1108:53xificurCthe starter app has it in its https://github.com/hyperfiddle/electric-starter-app/blob/2033b010393237c0b87c59c981fdd31155c05e3e/deps.edn :deps, along with a logging impl (logback)#2023-09-1108:55grounded_sageyea. I am a little confused as to how to resolve this. I've been looking at clj -Stree etc and looked at dependencies. It does build normally so somehow Datahike being added is booting it off the path?#2023-09-1108:57xificurCbtw clj -X:build:deps list to list deps with the :build alias#2023-09-1108:58xificurCdo you have a repo where I can repro your issue?#2023-09-1109:25grounded_sageWill have one to share later today#2023-09-1211:00grounded_sageSo I've gone back to a minimum repo to figure out how I actually build a persistent database connected Electric.
Here is the repo: https://github.com/groundedsage/electric-starter-datahike
Changes I made.
• Moved user.clj to src-dev
• added (alter-var-root #'app/!datahike (constantly (app/start-datahike!))) to prod.clj
• the hyperfiddle version check is commented out to test
clj -M -m prod works
clj -A:dev -X user/main works
When I build the jar and then run it. This is the error I get.
$ electric-starter-datahike % java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -jar app.jar
Exception in thread "main" java.lang.NoSuchFieldError: __thunk__0__
at clojure.tools.analyzer.jvm.utils__init.load(Unknown Source)
at clojure.tools.analyzer.jvm.utils__init.<clinit>(Unknown Source)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:496)
at java.base/java.lang.Class.forName(Class.java:475)
at clojure.lang.RT.classForName(RT.java:2209)
at clojure.lang.RT.classForName(RT.java:2218)
at clojure.lang.RT.loadClassForName(RT.java:2237)
at clojure.lang.RT.load(RT.java:449)
at clojure.lang.RT.load(RT.java:424)
#2023-09-1211:04grounded_sageThere is a little ceremony to run Datahike at present due to the branch I am using not being on main and the way Datahike is at present doesn't allow the usage of git sha. I haven't quite figured out how to resolve that because being able to use a git sha would be way more convenient for this kind of work.#2023-08-3108:14CanHello everyone, I'm looking for an idiomatic way to add that delay to button. For example, after the user clicks the button it will be disabled for 1 second to prevent multi-clicking etc. How can I show my query's result or alert message on the page with a popup or HTML alert? (I tried js/alert but was not able to progress on it, a popup doesn't appear)#2023-08-3108:19xificurCThe ui4 namespace contains a button implementation that disables itself and adds a spinner while the e/fn callback is pending (waiting for server to process your request). This is more resilient and correct than adding arbitrary delays#2023-08-3113:04Dustin Getz@U04SVJW7DLZ please paste the code you tried#2023-09-0313:24CanSorry about my late answer, @U09K620SG I need to re-write that code to send here.
(ui/button
(e/fn [] (js/alert "hello ")
)
(dom/text "alert!")
(dom/props {:class "alert"
:name "alert-button"
:style {:text-align "center"}
}))
Nothing happened after I clicked the button, I am not sure that I am using it in the correct way but in past Slack massages, I saw some similar usages.#2023-09-0314:24Dustin GetzLooks fine to me, a working button example can be copy pasted from here: https://electric.hyperfiddle.net/user.demo-toggle!Toggle#2023-09-0314:25Dustin Getztry println instead of js/alert#2023-09-0315:54Canprintln working nicely, I can see printed text on the console. How can I use HTML alerts or JS alerts in Electric. Thanks for separating time 🙂#2023-09-0315:55Dustin Getzare you on latest electric? what version are you using#2023-09-0315:55Can"v2-alpha-284-g0bc5f4bc"
#2023-09-0315:55Dustin Getzplease upgrade that is very old#2023-09-0315:56CanOh okey, let me try it. Thank you so much!#2023-09-0316:07Canat java.base/java.lang.Thread.run(Thread.java:833)
DEBUG hyperfiddle.electric-jetty-adapter: Client disconnected for an unexpected reason. {:status 1011, :reason Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null}#2023-09-0316:08Dustin Getzunfortunately you must upgrade#2023-09-0316:09Dustin Getzthe changelog documents breaking changes (may not impact you) and you'll need to validate your configuration against the starter app https://github.com/hyperfiddle/electric-starter-app#2023-09-0316:09CanI upgraded but the project doesn't work at the moment.#2023-09-0316:09Dustin GetzYou'll need to debug it unfortunately#2023-09-0316:10Dustin GetzPerhaps easier to start with a fresh clone of the starter app and copy your code in#2023-09-0316:10Canoh let me try to debug first 😞 Thank you so much.#2023-09-0316:11CanERROR hyperfiddle.electric-jetty-adapter: Websocket error#2023-09-0316:12Canthis is the error, I leaving this here because maybe someone knows it.#2023-09-0316:13Dustin Getzfirst thing to check is see if your entrypoint is different from the starter app#2023-09-0412:50Can@U09K620SG hello, https://github.com/hyperfiddle/electric-starter-app/blob/main/deps.edn is that version the latest one? I am going to clone it if yes.#2023-09-0413:24Dustin Getzyes this the latest starter and is the correct thing to clone#2023-09-0413:24Dustin Getzbump electric dep to v2-alpha-428 though#2023-09-0107:21henrikI’ve put Electric on http://Fly.io, which doesn’t respect WS heartbeats. It’ll close down the connection after 60s, which causes the UI to flicker, if idle. This can apparently be fixed by sending some random junk (that is not a heartbeat) down the wire every so often. What would be a good way to do this? Should I make a small Missionary machine that sends a random UUID or something like that, or could I put this lower in the stack somewhere?#2023-09-0107:27Geoffrey GaillardLatest Electric mvn release (`v2-alpha-428-g22937f75)` sends a heartbeat https://github.com/hyperfiddle/electric/commit/fdacafedf43b96db99ba7cf4403f931ab20e53a0#diff-11e66f20e43a2bb061b1385a8c9254e2b6a9663a4d0fc8aaadea932b4115ce4bR79. Are you running latest?#2023-09-0107:43henrikYes, but I’m also using Aleph, and have built an adapter for it. It sends a heartbeat every 35 seconds.#2023-09-0107:45henrikIt seems to be a Fly thing, not an Electric thing: https://community.fly.io/t/app-connection-timeout/5369#2023-09-0107:50Geoffrey GaillardIn that case yes, you’ll need to send text or binary. Fly seems to ignore WS PING frames.
A small missionary machine is the way to go.
We implemented one in https://github.com/hyperfiddle/electric/blob/ef937e253c1a453695f66dd93ff500cf661c9892/src/hyperfiddle/electric_client.cljs#L72.
It sends the https://github.com/hyperfiddle/electric/blob/ef937e253c1a453695f66dd93ff500cf661c9892/src/hyperfiddle/electric_client.cljs#L94 from client to server every 45 seconds.
The server adapter receives it, https://github.com/hyperfiddle/electric/blob/ef937e253c1a453695f66dd93ff500cf661c9892/src/hyperfiddle/electric_httpkit_adapter.clj#L67-L68#2023-09-0107:52henrikYou know what, I am seeing something weird though, with the latest version. Probably in relation to the latest version of Electric and Aleph:
2023-09-01T07:50:54.413Z Clavain.local ERROR [manifold.stream:?] - error in stream handler
java.lang.Thread.run Thread.java: 1623
...
manifold.executor/thread-factory/reify/f executor.clj: 70
io.aleph.dirigiste.Executor$Worker$1.run Executor.java: 62
...
manifold.deferred.Deferred/fn/fn deferred.clj: 397
manifold.deferred.Listener/onSuccess deferred.clj: 218
manifold.stream.graph/async-connect/this/fn graph.clj: 189
clojure.core/trampoline core.clj: 6345
manifold.stream.graph/async-connect/this/fn/fn graph.clj: 189
manifold.stream.graph/async-connect/this graph.clj: 208
manifold.stream.graph/async-send graph.clj: 82
manifold.stream.Callback/put stream.clj: 453
app.system.electric/aleph-open-connection/on-msg electric.clj: 85
clojure.core/comp/fn core.clj: 2586
app.system.electric/decode electric.clj: 49
io.cljc: 182
cognitect.transit/read transit.clj: 323
com.cognitect.transit.impl.ReaderFactory$ReaderImpl.read ReaderFactory.java: 112
com.cognitect.transit.impl.JsonParser.parse JsonParser.java: 43
com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken UTF8StreamJsonParser.java: 793
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject UTF8StreamJsonParser.java: 907
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue UTF8StreamJsonParser.java: 2781
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken UTF8StreamJsonParser.java: 3693
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError ParserMinimalBase.java: 759
com.fasterxml.jackson.core.JsonParser._constructError JsonParser.java: 2418
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'HEARTBEAT': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (ByteArrayInputStream); line: 1, column: 10]
location: #object[com.fasterxml.jackson.core.JsonLocation 0x370a73dc "[Source: (ByteArrayInputStream); line: 1, column: 10]"]
originalMessage: "Unrecognized token 'HEARTBEAT': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')"
processor: #object[com.fasterxml.jackson.core.json.UTF8StreamJsonParser 0x1ab7bf9a "com.fasterxml.jackson.core.json.UTF8StreamJsonParser@1ab7bf9a"]
java.lang.RuntimeException: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'HEARTBEAT': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (ByteArrayInputStream); line: 1, column: 10]#2023-09-0107:53henrikI’ll go back and look at your code, I probably must update the Aleph adapter a bit.#2023-09-0107:53Geoffrey GaillardYes, you’ll need to adapt your Aleph adapter to the “HEARTBEAT” message#2023-09-0110:18Dustin Getz@U2DART3HA i think this is a known bug in master that i triaged while you were on vacation , i have this behavior on all my apps currently #2023-09-0110:20Dustin Getzhenrik’s log is missing timestamps but if i am right he will see that this occurs every 5 minutes not every 1 minute#2023-09-0110:22Dustin Getzit happens on localhost also not just on fly for me#2023-09-0110:24henrik@U09K620SG Just adding the when-not to the Aleph adapter actually fixed it for me.#2023-09-0200:08weiI'm working on a electric/xtdb/fly.io starter example (https://github.com/yayitswei/electric-xtdb-starter-fly-io), currently troubleshooting an uberjar issue in this #xtdb thread: https://clojurians.slack.com/archives/CG3AM2F7V/p1693613079965599?thread_ts=1677430221.688989&cid=CG3AM2F7V. lmk if you have any insights, thanks!#2023-09-0200:37Dustin Getzmove user.clj off classpath uberjar build, put it in src-dev or something and adjust deps.edn accordingly#2023-09-0200:38Dustin Getzthis has been addressed in the starter app i believe or if not is about to be in the next version #2023-09-0200:38Dustin Getzi cannot explain the issue but search this channel for the NoClassDefError#2023-09-0219:25weithanks for the tip, that did the trick. working deployment at https://electric-xtdb-starter.fly.dev/#2023-09-0219:27weitook longer to troubleshoot because the cljs build was failing but the deployment continued. any good way to stop the build on cljs compilation failure?#2023-09-0219:30weifor that matter, i'm having a hard time seeing the full build log. I'm using NO_COLOR=1 but still seeing the truncated logs -- are there any other tricks to see the full log?#2023-09-0219:30wei#2023-09-0219:31Dustin Getzbuilds are getting redone, file any issues on the repo and i’ll confirm when we fan out the changes to all the starter repos#2023-09-0219:31wei(understand this is more of a http://fly.io question, i can ask around as well)#2023-09-0219:32weisg thanks!#2023-09-0219:33Dustin Getzi don’t really understand the question about build logs, i think we use github actions for our builds so we go there for the logs? is this a docker question?#2023-09-0219:33Dustin Getzfly sends runtime logs to any configured cloud logger service, but i don’t think build logs would go there#2023-09-0219:36weiiiuc the electric starter app is configured to build on a http://fly.io machine. i can see runtime logs on http://fly.io, but not build logs, it just flies by in that truncated format. i copied the electric starter app deploy configuration for the xtdb starter.#2023-09-0319:30Dustin Getz[deleted confused replies]#2023-09-0320:28Dustin GetzYou can use this snippet to fail the shadow build, until we get it landed everywhere:
(as->
(shadow-api/release :prod
{:debug debug,
:verbose verbose,
:config-merge
[{:compiler-options {:optimizations (if optimize :advanced :simple)}
:closure-defines {'hyperfiddle.electric-client/ELECTRIC_USER_VERSION electric-user-version}}]})
shadow-status (assert (= shadow-status :done) "shadow-api/release error")) ; fail build on error#2023-09-0320:28Dustin Getzyou'll need to adapt it to look like whatever your app has in build.clj, this has other changes#2023-09-0320:29Dustin GetzI have not fully tested it, iiuc an exception is sufficient but you can add a (System/exit 1) if CI needs us to "fail harder"#2023-09-0620:35weithat's helpful, thanks! i ended up running the build locally in docker to find out what the cljs compiler was complaining about#2023-09-0317:46Dustin GetzAnyone who is overriding *ws-server-url* I am making changes to how this works (to auto-wire the client/server version check in the connect string), please reply here if you have customized this and paste me a snippet showing me how you did it#2023-09-0318:38grounded_sage👋😅#2023-09-0318:42grounded_sage(set! client/*ws-server-url* "")
I am script tagging on a different domain and client/*ws-server-url* appeared to be pulling the current host domain. Was unclear how this could be done another way.#2023-09-0321:32grounded_sageI had also changed the string name as I didn't want people knowing what we were using under the hood. But if they are already smart enough to go into devtools they could figure it out.#2023-09-0322:45Garrett HopperI have it overridden for storybook, but it's only necessary in Dev of course, so the client version is blank.#2023-09-0318:44grounded_sageHas anyone got a code snippet of debouncing user text input with missionary? I still haven't grokked it and see references to m/observe and m/relieve when I search debouncing in the chat here.#2023-09-0318:45Dustin Getzdebounce is a UI anti-pattern due to introducing latency artificially (never slow down the user from doing what they want)#2023-09-0318:45Dustin Getzplease state the problem you're trying to solve?#2023-09-0318:51grounded_sageI am doing a text search on the database. So was going to debounce the calls by around 150ms and check if there is new results.
Two reasons:
1. Not querying the database on every single keypress
2. Currently the UI flickers on each added or removed character#2023-09-0318:56Dustin Getzthe UI flickering indicates something else bad is happening, that should be fixed separately#2023-09-0318:56Dustin Getzto throttle the database i recommend putting the throttling code around the database, not around the user input#2023-09-0318:58Dustin Getzper the other discussions you found via search, you want to allow the first request through and cancel it if new requests arrive that supersede the old request#2023-09-0318:59Dustin Getzif the database supports cancellation (respects ThreadInterruptException which missionary will raise when the query call has been superseded by a newer query call), there's nothing you need to do, and you definitely do not need to add delays#2023-09-0318:59Dustin Getzthe old superseded query will be interrupted/discarded and the new query takes precedence#2023-09-0319:00Dustin Getzif the database doesn't support cancellation (does not respect ThreadInterruptException) then what will happen is the query will need to finish before serving the next query#2023-09-0319:00Dustin Getzin this case i dont think there is anything for you to do either#2023-09-0319:00Dustin Getzit will just be slower because the database is failing to respect the cancellation signal and therefore is making the clients wait for delivery of a query result that is irrelevant and will be ignored/discarded#2023-09-0319:01grounded_sageThis is incredibly valuable info.#2023-09-0319:02grounded_sageI will investigate further into my UI flickering. I thought it might have been my error but then thought it could be something to do with the database interaction.#2023-09-0319:02Dustin Getzin this slow case, due to continous time work skipping, if you button mash the text input, missionary (electric) will skip all the intermediate inputs and jump straight to the latest value of the input#2023-09-0319:03Dustin GetzPer my current understanding of UI programming in Electric, the only time you'd ever want to artifically throttle something is a side effecting call like d/transact#2023-09-0319:03Dustin Getzif you smash Datomic d/transact rapidly, it gets confused and crashes or hangs or something due to concurrency bugs in Datomic, I forget the details#2023-09-0319:04Dustin Getzthat is a case where adding artifical latency is sensible – to work around broken dependencies#2023-09-0319:05Dustin Getzin Datomic's defense, it was not designed for d/transact to be rapidly invoked, so it is abusive for a reactive app to use it like that. (Nonetheless, databases should not be crashing)#2023-09-0319:06grounded_sageI will definitely see about this being added to Datahike. I'm glad I have an opportunity to really use all these tools and see where they can work better together.#2023-09-0319:06Dustin GetzSo in the case of transacting rapidly into a database, and assuming that it's not an abuse of the database, the stream of effects can be throttled with missionary to not overwhelm it#2023-09-0319:59grounded_sageOkay this UI flicker I am uncertain on. All of this is in a let binding.
!search-text (atom nil)
search-text (e/watch !search-text)
conversations (if-not search-text
(e/server
(sort-by first > (d/q '[:find ?created ?e ?conv-id ?topic
:where
[?e :conversation/id ?conv-id]
[?e :conversation/topic ?topic]
[?e :conversation/created ?created]
[?e :conversation/folder :default]]
db)))
;; This server call causes a UI flicker
(e/server
(let [convo-eids (d/q '[:find [?c ...]
:in $ search-txt ?includes-fn
:where
[?m :message/text ?msg-text]
[?c :conversation/messages ?m]
[?c :conversation/topic ?topic]
(or
[(?includes-fn ?msg-text search-txt)]
[(?includes-fn ?topic search-txt)])]
db search-text lowercase-includes?)]
(sort-by first > (d/q '[:find ?created ?e ?conv-id ?topic
:in $ [?e ...]
:where
[?e :conversation/id ?conv-id]
[?e :conversation/topic ?topic]
[?e :conversation/created ?created]
[?e :conversation/folder :default]]
db convo-eids)))))
Input
(dom/input (dom/props {:class "w-full flex-1 rounded-md border border-neutral-600 bg-[#202123] px-4 py-3 pr-10 text-[14px] leading-3 text-white"
:placeholder "Search..."
:value search-text})
(dom/on "keyup" (e/fn [e]
(if-some [v (empty->nil (.. e -target -value))]
(reset! !search-text v)
(reset! !search-text nil)))))#2023-09-0320:03grounded_sageThe one with a single query does not flicker. The one with the two queries does flicker.
The results are sorted by date before being returned from the server. As I type the flicker occurs when nothing changes in the results.#2023-09-0320:05Dustin Getztry e/offload around any heavy lifting (that would include the sort)#2023-09-0320:17grounded_sagehmm I still seem to get a flicker. I also tried it with m/cpu
playing around with variations on the code to see if there is more info I can provide.#2023-09-0320:17Dustin GetzWe will probably have to take a look, flicker isn't something I have seen a lot#2023-09-0320:23grounded_sageokay np#2023-09-0410:03CanHello everyone, I have a question about how advanced routing system works in Clojure / Electric.
Can you take a look and if possible help me please?
https://stackoverflow.com/questions/77036370/how-parametered-routing-system-works-in-clojure-hyperfiddle-electric-dsl#2023-09-0413:18telekidI haven't anything close to a canonical routing story for electric apps (nor would there necessarily be?) Have you come across https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc and https://github.com/hyperfiddle/electric/blob/master/src/contrib/sexpr_router.cljc? I believe the latter is used to store app state in the URL for https://electric.hyperfiddle.net/.#2023-09-0413:19telekidI'm prototyping an electric-focused router called https://github.com/telekid/pondo, but I wouldn't use it yet (and there are no docs)#2023-09-0413:25Dustin Getz@U04SVJW7DLZ the hyperfiddle.router ns is experimental, we make breaking changes, i don't recommend it for beginners you have to read the source#2023-09-0413:27Dustin Getzif you still want to use it despite being immature, you can learn by example from
• https://github.com/hyperfiddle/electric-datomic-browser
• https://github.com/hyperfiddle/electric-examples-app (http://electric.hyperfiddle.net)
#2023-09-0413:27Dustin GetzYou can also try using core.match for the router now that we support core.match#2023-09-0413:29Dustin Getzin combination with contrib.electric-goog-history (https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc), which is very simple, 20 LOC, probably the minimum viable history integration#2023-09-0413:30Dustin Getzcontrib namespace also may have breaking changes in the future, sorry. But you can always copy/paste this and maintain it yourself as it is 20 LOC#2023-09-0415:14KeinI have a function:
(e/defn create-data! [data]
(e/server (d/transact! !conn [ (Transform (Query. data)] ) ;The Query. is an e/fn
)
With its context:
(e/defn Home []
(Event-System.)
)
(e/defn Event-System []
(let [event (e/watch !event)]
(when (= (:status event) true)
(Dispatch-Event. (:event event))
(reset! !event event-default))))
(e/defn Dispatch-Event [event]
(let [event-map {:create-data create-data!}]
(new ((:type event) event-map) (:edata event))
)
)
The create-date is supposed to be reactively computed when I press “Enter” , but instead there is no effects, i.e. the transaction did not execute. [0]#2023-09-0415:15KeinWhat confuses me is:
[1] When I change the code to the following, and presss “Enter”, it worked , showing “ASD” in JS console. This means that the code could be executed.
(e/defn create-data! [data]
(println "ASD")
)
[2] In the REPL I tested the following, successfully mutate the state, and get the correct result showing in the UI.
(d/transact! !conn [ (Transform (Query data)] ) ;the Query is a non-e/fn normal query
[3] When I hook the transaction in an e/fn inside an e/button (as the following), the transaction ran, but 2 items in my todo-list disappeared, and when I refresh the browser , no data can be queried.
(e/defn Send [] (ui/button (e/fn []
(create-data!. {:uid "test7" :value "test13"})
)
(dom/text "Send")))
[3.1] Both Datalevin and Electric have no runtime error.
[4] After [3], the “/tmp/datalevin/mydb” is broken. When I query and transact, there is no response.
[5] After [4], I reload the page 2-3 times and hit the ‘transact’ button 2-3 times , the memory usage increased from 1GB to 4GB and crashed the system.#2023-09-0415:21KeinI’m not sure whether there is some un-idiomatic process, and I’m not sure why [1]-[5] happens.#2023-09-0415:26xificurCwhat is Transform?#2023-09-0415:29xificurCI don't understand the purpose of Event-System but it sets my spidey senses tingling. Electric is a continuous time language whereas events are discrete. If you expect !event to act as a sort of event stream note that you will hit a case where electric skips some of them#2023-09-0415:30xificurCmaybe your ultimate goal can be accomplished by other means. What problem are you trying to solve?#2023-09-0502:46KeinThe create data receives {:target-id x :value y } , and Transform turn this data into a datalog transaction body : [ {:db/id -1 , :value y } { :db/id (Get-ID x)} :next -1 ]#2023-09-0502:48KeinThe purpose of the event system is to do e/fn inside reagent component. I find no way to hook e/fn inside the event-listener in reagent component. So I have to cache the event in a global atom. Is this an idiomatic way? Maybe there’s another approach?#2023-09-0502:48KeinIn Dispatch-Event, the event-map is something that we register new function there.#2023-09-0506:57xificurCis Transform a clojure or electric function?#2023-09-0506:59KeinA clojure function , written locally inside create-data!#2023-09-0507:02xificurCregarding reagent components, we haven't settled on a singular solution but hepled users on a case by case basis. If the events you care about are DOM events the simplest approach is
(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))
#2023-09-0507:05xificurC> A clojure function , written locally inside create-data!
I'm asking because we titlecase electric functions to disambiguate them from clojure functions.
create-data! is an electric function, is Transform defined inside the e/server block?#2023-09-0507:07KeinTransform is not a function, but an abbreviation of some operation inside e/server#2023-09-0507:08KeinIt’s the following:
(e/server
(d/transact! !conn
[{:db/id (:db/id (Get-Node-From-ID. "test7"))
:next -1}
(merge
{:db/id -1,
:user "Kein",
:uid "test13",
:parent (:db/id (Get-Node-Relation. "test7" :parent)),
:value "test13"}
(if-let [x (:db/id (Get-Node-Relation. "test7" :next))] {:next x} {}))]))#2023-09-0507:08KeinThe Get-Node-From-ID and Get-Node-Relation is abbreviated as Query.#2023-09-0507:11KeinI think the primary difference between our approaches is that, I use a normal watch and a normal function Create-Data! , and you hook an extra e/fn in dom/on. Are there any difference between the 2?#2023-09-0507:39xificurCI think I know what the problem is.
> I use a normal watch and a normal function Create-Data! , and you hook an extra e/fn in dom/on
The reason I want to use dom/on is it has known semantics, e.g. I can tell what happens if a new event arrives while the previous one is still executing on the server. The same can be said about the ui4 controls, which are not final but have been ironed out to have reasonable semantics at this point. So e.g. a ui4/button is a transcactional button, it's disabled while the server is processing a request.
Generally speaking electric is a concurrent language and building these little machines can be tricky. Looking at your event system implementation
(e/defn Event-System []
(let [event (e/watch !event)]
(when (= (:status event) true)
(Dispatch-Event. (:event event))
(reset! !event event-default))))
Dispatch-Event and reset! run concurrently. What happens in your case is
• Dispatch-Event runs create-data!, which has a server block, so it throws Pending
• concurrently reset! changes !event, the watch fires
• now I guess when returns false, unmounting Dispatch-Event
• therefore create-data! unmounts as well, before the server had the chance to finish
I'd advise to use dom/on and the ui4 namespace where possible. If you're interfacing with a reagent component and can't use ui4 and dom/on doesn't have the correct semantics please describe the semantics you wish to achieve and I can help building it#2023-09-0514:01Kein@U09FL65DK Regarding:
(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))
[Q1] What can this-div be, any level of reagent-component ? Such as:
[:> Slate
{:editor editor
:initialValue [{:children
[{:text (:value data)}],
:type "paragraph"}]
:onChange (fn [value])}
[:> Editable ]
] #2023-09-0514:03Kein[Q2] And should we do a lot of (dom/on) for different events or even different kinds of div? Do you have a file containing an implementation of this pattern (or more context) ?#2023-09-0514:11Kein[Q3]
(e/defn Event-System []
(let [event (e/watch !event)]
(when (= (:status event) true)
(Dispatch-Event. (:event event))
(reset! !event event-default))))
When !event change, which functions run, and which runs first?
[A] when
[B] Dispatch-Event
They all have event which is watched.#2023-09-0514:13xificurCQ2 was discussed several times in the channel, here's probably https://clojurians.slack.com/archives/C7Q9GSHFV/p1683292217371419. I'd do it as if it was in electric, per DOM node#2023-09-0514:13KeinOK Thanks#2023-09-0514:26xificurCQ1 I'm not sure what Slate returns, it's a react wrapper that probably builds a lot of DOM nodes. You'll have to poke around. Options I see
• the https://docs.slatejs.org/libraries/slate-react/event-handling are normal DOM event handlers installed on a top-level DOM node, in which case you can use the trick described above
• the events/callbacks you want to handle are not plain DOM event handlers. Here it gets trickier and you'll need to bridge to electric differently, probably through m/observe and using one of the event handlers from electric such as e/for-event
• there's a non-react slate package, maybe it'd be easier to create a native electric wrapper?#2023-09-0514:27xificurCif you end up with the second option you could create a repo I can clone and start easily and show me where you got stuck and I can flesh out the initial implementation for the bridge#2023-09-0514:29xificurCQ3 I think when will run first, but I'm not sure. You can add some printlns to double check#2023-09-0514:39Kein[Option1 (O1) -Q1] Peter, what do you mean by normal DOM event handlers and not-plain DOM event handlers?
[O1-Q2] The trick is the following, right? [Q2.1] So you mean that we do the reset! in the normal event handler? [Q2.2]
(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))
#2023-09-0514:45xificurChttps://docs.slatejs.org/libraries/slate-react/event-handling say
> By default, the Editable component comes with a set of event handlers that handle typical rich-text editing behaviors (for example, it implements its own onCopy, onPaste, onDrop, and onKeyDown handlers).
If these are equivalent to basically document.getElementById('editor').addEventListener('keydown', e => ...) then I mean they're "normal". If they are called onKeyDown but don't map 1:1 to the DOM keydown event then they're "not plain"#2023-09-0514:54KeinI see. I’ll take a look at the m/observe and others for a while. Thanks a lot 🙂#2023-09-1913:15KeinI tried bridging reagent with electric using for-event-pending-switch provided by Peter. But finally it turned out that the issue is from Query. . I should let the Query function be normal instead of an e/fn. It seems that if I use e/fn, the function will keep updating while I want it to be computed only once. Or are there any better explanations?#2023-09-1913:17xificurCyou should write electric functions when you require client/server transfer#2023-09-1913:18xificurC> if I use e/fn, the function will keep updating while I want it to be computed only once
that probably isn't the case, (q x) and (Q. x) will both react when x changes#2023-09-1913:25KeinAs long as (q x) is defined in an e/defn , (q x) is reactive, right?#2023-09-1913:42xificurCwhat do you mean by "defined in an e/defn"?#2023-09-1913:45KeinIn an e/fn , every expression is a DAG node, which means that the return expression and all its dependent expressions are DAGs#2023-09-1913:52Kein[Q4] So in an e/fn, if both (q x) and (Q. x) are dependencies of the return value, they are both reactive? I.e. in the following :
(e/defn Fn1 [x y]
(fn2 (q x) (Q. y))
)
[1] When x change, q is recomputed , if (q x) is different then the previous result , Fn2 is recomputed
[2] When y change, Q. is recomputed , if (Q. x) is different then the previous result , Fn2 is recomputed#2023-09-1914:05xificurCcorrect#2023-09-0415:34nakkayaI have a map with the following structure
{coll-id {query-id-a {:query "<some query>", :response ["list" "of" "tokens"]}
query-id-b {:query "<some query>", :response ["list" "of" "tokens"]}
query-id-c {:query "<some query>", :response ["list" "of" "tokens"]}}}
the :response is streamed from a LLM model, each new token generated
is appended to the :response vector. Then on electric side I have,
(try
(e/server
(e/for [[_ q] (responses coll-id)]
(e/client
(dom/div
(dom/props {:class "mt-3"})
(dom/h4
(dom/text (:query q)))
(dom/div
(dom/text (apply str (:response q))))))))
(catch Pending _))
During the update the latest response (the one getting the updates)
text is flickering while older responses are staying solid. Is there a
work around to avoid flickering?#2023-09-0415:43xificurCcould the cause of flickering be that the DOM nodes are unstable? If so you need to make sure e/for or e/for-by is stable across runs of responses#2023-09-0415:44xificurCI think (e/for-by first ...) will do the trick#2023-09-0415:45xificurCright now e/for is stabilizing on the whole [_ q] where the latest response is still changing. If the first part is stable you need to use it as the e/for-by key#2023-09-0415:54nakkaya> If so you need to make sure e/for or e/for-by is stable across runs of responses
Can you expand a little bit on what this means? In my example case values for query-id-a and query-id-b are not changing. Only the :response in query-id-c is updating. This is the place where the flicking happens. div s for the first two items stay solid as do the :query part of query-id-c . But the :response part is flickering with each redraw. (if this would be the correct terminology.)#2023-09-0418:05xificurCSure! e/for-by manages values for you. Each unique value is rendered separately. When your query re-runs it needs to decide which renderings need to be added, updated or deleted. This is e-for-bys first argument, a key-fn, which e/for defaults to identity. If (= previous (key-fn current)) it's an update, if not it is an add/delete.
BTW this is similar to react.js key#2023-09-0420:40nakkaya@U09FL65DK thanks for the explanation, that worked. I was wondering why the old scheme did not flicker but the new one did.#2023-09-0421:13Dustin Getze/for should just never be used, we will remove it#2023-09-0416:06CanHello, I am faced with a strange error. While I developing one page, I clicked a button in the browser then suddenly I got this error and now I getting the same error.
What I did:
- Restarted pc
- Ofc undo
- I recloned the newest electric version and implemented my code result is unfortunately the same.
- I deleted DB and recreated but nothing fixed.
- I deleted the last page which I working on but still getting the same error.
Until I click that button everything was working nicely and without problem.#2023-09-0416:25Dustin GetzTo debug this I recommend commenting out parts of your app to try to triangulate which piece of code in your app is causing the problem#2023-09-0416:25Dustin Getz(Thank you for upgrading electric versions, that gives us a common baseline point)#2023-09-0416:26Dustin GetzAh, you stated that it is the button that causes the problem#2023-09-0416:27Dustin GetzCan we comment out pieces of the e/fn in the button to figure out which expression breaks it?#2023-09-0416:27CanHello, I solved but needed to delete 3 pages 🙂#2023-09-0416:27Dustin GetzI think you can triangulate it better than 3 pages#2023-09-0416:28CanI deleting my post thank you so much for separating time.#2023-09-0416:28CanI deleted single by single which didn't solve the problem, I tried to delete group by to find errors scope then I found out that it works if I delete like that.#2023-09-0418:48joshchoI have a fairly complicated defn that I don't want to turn into e/defn. I want to be able to do a blocking call with the server, where I ask for some value from the server and the execution doesn't move along until it get it.
Also, I would be open to changing it to a e/defn, but sometimes reactivity makes things not so straightforward.#2023-09-0418:49Dustin Getzcan i see sample code#2023-09-0418:50joshchoprocess-stack is the function I am keeping as defn#2023-09-0418:51joshchoprocess-stack takes in stack (for use in Forth) and a value (which is just a string like "dup dup"), and updates the stack. The third argument realize? is there so that calling with realize? = true actually does the thing, whereas if it's false, only a preview of the result is shown.#2023-09-0418:52Dustin Getzcan you re-upload that as an attachment please, otherwise cannot view this thread in mobile#2023-09-0418:52joshcho#2023-09-0418:52joshchodoes that work?#2023-09-0418:52Dustin Getzyes#2023-09-0418:53Dustin Getzset filetype to clojure please as well#2023-09-0418:54Dustin Getzwhat is cmt p/peek etc#2023-09-0418:54joshchoOh those are just logging functions#2023-09-0418:54joshchop/ is logging , tw is tailwind#2023-09-0418:54Dustin Getzif you can reduce the LOC by removing noise to make this easy for me that would help, which line is the blocking line?#2023-09-0418:55joshchoI can try removing some stuff if that will help (I sometimes end up removing too much)#2023-09-0418:55joshchoyeah#2023-09-0418:55Dustin Getz`
(enter
(e/fn [s]
(when-not (re-matches #"\s*" s)
(process-stack stack s true))
(reset! !value "")))
#2023-09-0418:56Dustin Getzyou said it's supposed to run on the server?#2023-09-0418:56joshchoNo, all of this right now runs on the client, but I want to extend this to be able to have some parts run in the server#2023-09-0418:57joshchoprocess-stack right now is bound to client atm#2023-09-0418:57joshchoI can't get it to work when I change it to e/defn#2023-09-0418:57joshchoIt works for enter, but doesn't work for preview#2023-09-0418:58joshchocleaned up Main#2023-09-0418:59Dustin Getzgeneralizing, if you have a structure like (case (e/server (foo ...)) (println ::done)) the conditional node case will be forced to wait for the pending value to resolve to know which branch to run. You can exploit that in this way, like here, to run the println after the remote value has resolved#2023-09-0418:59Dustin Getzyou could use if-some for example to wait for the value and then continue into the body with the result bound#2023-09-0419:02joshchoIIUC, the problem rn is that I can't put e/server in because the process-stack function is just a defn. Ideally it would become a e/defn, or do a non-Electric thing within the defn. There's probably something wrong with my Main function logic#2023-09-0419:03Dustin Getzyou'll need to perform the e/server in an e/defn and pass in those values to the clojure fn#2023-09-0419:03joshchothe e/defn version#2023-09-0419:03Dustin Getzalternatively you can call the clojure function on the server and then do any clojure stuff (like block) inside that function#2023-09-0419:03joshchoThe waiting for values are not trivial, it's recursive and intertwined together.#2023-09-0419:03joshchoThat could work#2023-09-0419:04joshchoIs there something wrong with the way I am using process-stack here? The first process-stack. in Main isn't called when value updates. (value and stack are watches).#2023-09-0419:06joshchoI also tried something like
(new
(e/fn [stack value]
(process-stack. stack value false))
stack value)
but doesn't quite work.#2023-09-0419:23joshchoI resolved it using Missionary, but maybe isn't necessary.
(e/for-event
[v (e/fn [] [stack value])]
(process-stack. stack value false))
#2023-09-0507:13xificurCcould you explain what the goal is? What do you wish to move to the server in process-stack?#2023-09-0419:02nivekuilwhy does ui/input lose focus every time it re-renders? code for reference https://gist.github.com/nivekuil/5518e363785763f1c1bbf4d514594a0e#2023-09-0419:05Dustin Getzour live demos demostrate that in fact this does not happen#2023-09-0419:05Dustin Getzin your code if editing? turns false, the input will unmount, is that what is happening?#2023-09-0419:05nivekuilstill happens even I get rid of the if#2023-09-0419:06Dustin Getzi'll need to reproduce#2023-09-0419:06nivekuilok, I'm just 20 minutes into hacking the starter app, file as is#2023-09-0419:06nivekuilwith net.sekao/odoyle-rules {:mvn/version "1.3.0"}#2023-09-0419:06Dustin Getztry to repro without that#2023-09-0419:07nivekuilwithout what?#2023-09-0419:07Dustin Getzadding third party stuff#2023-09-0419:10Dustin Getzthe e/for does not have a key, is that the issue?#2023-09-0419:11nivekuilah#2023-09-0419:11nivekuilyeah#2023-09-0419:14nivekuilwhy does it happen with only 1 element though?#2023-09-0419:24Dustin Getzthe rules engine is returning a new reference with [possibly] equal value#2023-09-0419:26nivekuilgot it#2023-09-0419:26Dustin Getzwe never shoulda published e/for, this is a foot gun#2023-09-0419:26Dustin Getzit will be removed in electric v3 probably#2023-09-0501:42PanelWhat should be used instead of e/for ?#2023-09-0506:16henrikPresumably e/for-by identity, forcing you to be explicit about wanting whole value equality comparisons.#2023-09-0511:07pezHow do I test greater/less than with RCF? I tried :<, but that clearly wasn’t it. And checking the source code there doesn’t seem to be any other comparison “operator” available than :=. I’m probably thinking the wrong way about the whole thing.#2023-09-0511:19xificurConly := and :throws for now, sorry. You can (< x y) := true, which gives a vague true not false failure report so it's not perfect but works. There's work in progress in RCF to improve this but currently no time left to finish#2023-09-0511:20pezThanks. I’ll make do with what I have. 😃#2023-09-0517:01telekidUpgrading from v2-alpha-349-ge9996713 to v2-alpha-428-g22937f75, I get a Failed to load user.cljs RangeError: Maximum call stack size exceeded error. The issue appears to be exactly that: I can make the error appear and disappear by adding and removing an empty (dom/div) at a particularly deep (but not unreasonably deep!) leaf node in my application.#2023-09-0517:03telekidThe number of nested components is what I believe to be fairly typical for a web-app, I'm not doing anything wild here#2023-09-0517:26Dustin Getzwhat web browser#2023-09-0517:26telekidchrome#2023-09-0517:34telekidhappy to give you access to a branch on github if that would be helpful#2023-09-0517:38Dustin Getzyes please add me#2023-09-0517:45Dustin Getzwhat sha do i check out to repro by adding a single div and at which line#2023-09-0517:52telekid1. README at https://github.com/hypo-sh/hypo/tree/main/server shows command to launch nrepl server with cider + shadow middleware; then connect as normal
2. Start app by calling (user/main) and nav to http://localhost:8080/ui/ , then view console. You shouldn't see any errors
3. Upgrade hyperfiddle version to current version here: https://github.com/hypo-sh/hypo/blob/main/server/deps.edn#L25, then restart and reconnect repl and restart app
4. Check console; you should now see call stack size error reported
5. Comment out https://github.com/hypo-sh/hypo/blob/2287a11934b01eedc55577cd72aeff0091b6b1a9/server/src/hypo/server/ui/recording.cljc#L224-L228 and save to see issue go away. Replace that form with a simple (dom/div) to see the call stack size error return#2023-09-0518:01Dustin Getzperfect thank you#2023-09-0518:43Garrett HopperJust ran into this with the issue being turned on/off by adding/removing a core.match dependency, FWIW#2023-09-0606:27caleb.macdonaldblackI’m running into this one as-well in chrome.
I have a bunch of nested dom components
I noticed it with v2-alpha-422-g96e5c0a5#2023-09-0612:17Dustin GetzQuick update - we're escalating this issue, we first saw it several weeks ago on our largest project but didn't expect external users to hit it so soon. We don't expect it to be hard to deal with. Please continue to report any new information.#2023-09-0615:12xificurC@UABU2MMNW can you add me to your repo as well? Since you've gone through the trouble of preparing a repo with complete reproduction steps (much appreciated!) I'd like to use it for driving the fix#2023-09-0616:07telekidgladly, dm me your github email#2023-09-0616:08telekidnvm, found you#2023-09-0817:59xificurCupdate: master (unreleased) contains an optimization that reduces bundle size by ~15% on electric demos. If people could try it out to see if it resolves the stack size error that'd be very helpful!
@UABU2MMNW I tried it in your repo but I couldn't reproduce the initial stack size exceeded error locally after bumping to v428 🤷#2023-09-0819:34Dustin Getz@U797MAJ8M#2023-09-1200:32telekid> I tried it in your repo but I couldn't reproduce the initial stack size exceeded error
ah, strange#2023-09-1419:30Dustin GetzWe pushed changes related to this to master, if you have this error please try out the commit#2023-09-1513:53telekid@dustingetz can confirm, resolved the issue in my project. (I made sure to reproduce the issue on v2-alpha-428-g22937f75 prior to upgrading, too.)#2023-09-0520:42grounded_sageI'd like to understand a couple of things. @dustingetz I saw in messages here you saying e/for is kind of a footgun. What are the problems that people hit with this?
Also I am wanting to understand the usage of e/offload more. With the addition of Datahike is seems to require it for every transact . Is the assumption that you want to have this for everything that isn't reading data?
It also seems to cause some repeating transactions when using some values handled by e/watch against the db connection. I suspect this is a user error but it's not expected behaviour. Especially since those transacts are inside of an (dom/on "click" ...)#2023-09-0521:01Dustin Getz• e/offload moves a blocking computation to a threadpool (see docstring), so as not to block the reactive computation. databases are slow and blocking (even transact) so all database operations must be offloaded#2023-09-0521:01Dustin Getz• e/for-by should be used instead of e/for for all use cases. e/for simply macroexpands to (e/for-by identity ...) (see source code). It's a foot gun because identity is rarely the right "react key", and if you provide the wrong keyfn electric will rebuild the children of the for loop too much which can cause flicker and loss of local state
#2023-09-0521:02Dustin Getz• repeating transactions are probably caused by cycles in the "DAG". A common form of accidental cycle is a side effect on a reference (atom or database) which causes queries to refresh which causes the side effect to happen again. This interacts badly with how the electric-ui "callback" patterns work. The next iteration of electric-ui does not use callbacks at all to avoid this and other problems. In the meantime use e/snapshot to break cycles. search this channel for e/snapshot to see past discussions#2023-09-0601:00grounded_sagesnapshot patched that problem. Thanks!#2023-09-0601:01grounded_sageso e/offload queries as well. That was what I thought.#2023-09-0601:33grounded_sageI will have to delve a little more into the for. That might be what is causing my flicker. #2023-09-0609:51grounded_sageSnapshot fixed the flicker issue 🙌#2023-09-0609:57grounded_sageSpoke to soon. Not quite the fix I need.#2023-09-0600:10nivekuilelectric is working as expected here, but I thought it'd be interesting to share a problem I ran into with using odoyle for highly dynamic (ui-triggered, conditional, fine-grained, not subscribing to every database update) subscriptions:
1. button changes some data
2. data triggers subscription, which changes UI rendering path, creating a new child subscription
3. that new subscription has no data, because the data which would have triggered it was inserted in 1.
odoyle has no way of asking for data that was inserted before the subscription was added, as RETE is strictly for forward inference. My plan is to duplicate writes to asami for backward inference, and when the continuous flow is initialized it queries asami first (conveniently with the same datalog triple syntax), solving the empty initial state problem and allowing ad-hoc queries in general#2023-09-0600:18Dustin Getzthat’s what materialized does as well, an initial batch query and then differential subscription from there#2023-09-0600:19nivekuilyeah, it took me some thinking to realize that these are two completely different problems#2023-09-0613:06telekidAh, that's such a smart pattern. It steps around the event sourcing rehydration problem in a very elegant way.#2023-09-0614:34Dustin Getzthe other solution is the rama / golem.cloud / dataflow solution, where you make the language state itself durable and all changes to prod are "over the air updates". I dont know how Rama does it; i know Golem intends to restrict the set of legal code changes to those that don't require historical replay, and otherwise force you to write a state migration#2023-09-0621:57nivekuilasami not supporting upserts was a little annoying, but it still was pretty elegant, coming out to ~70loc including debug lines. now to figure out how to replicate writes across the network and have server->client reconcillation#2023-09-0621:59Dustin Getzfor others, i had to look it up https://github.com/threatgrid/asami#2023-09-0615:36Garrett HopperFYI, cljdoc build failing due to missing core.async: https://app.circleci.com/pipelines/github/cljdoc/builder/45561/workflows/e16a6237-979e-40af-b1b1-6cff4ff2608d/jobs/61936#2023-09-0615:36Garrett Hopperhttps://cljdoc.org/d/com.hyperfiddle/electric/v2-alpha-428-g22937f75/doc/readme#2023-09-0615:37Dustin Getzyeah we know, we tried to fix it when we merged your PR but failed and ran out of time#2023-09-0615:37Garrett HopperOk, cool
At least it's on the radar 🙂#2023-09-0615:37Dustin Getzwe will try again soon, unless you know the answer#2023-09-0615:37Dustin Getzi believe we tried :skip-doc or some such#2023-09-0615:38Garrett HopperI do not unfortunately; may take a look at some point, but it'll be a few weeks#2023-09-0615:59pezWhen ChatGPT got electrified 😃 https://twitter.com/pappapez/status/1699450096063029537#2023-09-0616:09pezIt thinks I should get started supporting Electric better in Calva. 😃#2023-09-0618:21teodorluInteresting! I wonder if it's picking up on your enthusiasm - and if that's why it's asking follow up questions.
If the same content was provided without joy, would it still want to know more?#2023-09-0618:43pezCould be, but I’m a joyful kind of person. If this would trigger it’s curiosity I should have been interviewed before. Could be a combo. I remember it was also very impressed with Missionary, when I told it about that the other day. But there I fed it the README and various documents and code examples, so it draw it’s own conclusion from that instead of relying on mine (which are probably terribly flawed anyway).#2023-09-0618:53jjttjjI was actually impressed with how well chatGPT caught on to electric once I basically just told it I was using a "clojure dom library called electric" and copy pasted the code i had then asked it for changes.
It was probably the best results I've had with "I'm using a clojure library to do X, here's some examples and here's what I want to do"
It's probably just that it understands html/the dom really well and electric is close enough that it doesn't struggle with the conversion. But I've definitely noticed it struggling more with trivial conversions in all other domains.#2023-09-0619:08pezI should start a thread where I feed it all I can find about Electric, like I did with Missionary.#2023-09-0618:01telekidhttps://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_compiler.clj#L10-L41, can't believe I hadn't come across it before#2023-09-0618:29telekidlow-importance question, what are the differences between current electric and upcoming differential electric?#2023-09-0619:00leonoelthat's not a low-importance question !#2023-09-0619:15leonoelThe general principles will stay the same. New features are :
• ability to inject an arbitrary differential collection. If the database supports it, that means zero runtime diffing all the way through browser.
• all expressions are multiplexed by default, so e/for-by is not bound to a single peer anymore.
• better performance due to reduction of network round-trips#2023-09-0619:37Dustin GetzDid that answer your question @UABU2MMNW?#2023-09-0619:42Garrett Hopper> If the database supports it
Curious if there's any known options at the moment that would support this?#2023-09-0620:20nivekuilif they're prioritizing this I wonder if they're planning to build a db on top of electric#2023-09-0620:24Dustin GetzRama, Materialized, anything streaming#2023-09-0713:14telekid> all expressions are multiplexed by default
What does it mean for a collection to be multiplexed?
> ability to inject an arbitrary differential collection
I'll take a stab at this one. In this case a collection might be a result set from a database, and a differential collection might be a result set from a database that can report fine-grained changes to the result set reactively – basically, a notification that members have been added or removed from the result set.#2023-09-0713:26telekid> build a db on top of electric
A logical next step, but also more of a leap than a step 😆#2023-09-0713:41leonoel> What does it mean for a collection to be multiplexed?
Multiplexed expressions. Each expression implicitly works on items of a list, as if it returned multiple values.
(e/client
(e/render-table
(e/render-row (e/server (e/inject result-set)))))
In this example e/inject takes a differential collection and returns all items in the collection. All derived computations also run on every item, it can seamlessly traverse the network, the ui layer can translate it to dom operations, etc.#2023-09-0713:42Dustin Getz"multiplexed" is basically amb from sicp#2023-09-0713:43leonoelyes, amb is the special case where collection has fixed size#2023-09-0713:47xificurCI'll tell you a story! When designing the next ui namespace we wanted to tackle a seemingly simple thing - render a todo list. When you have a todo-list UI you have an input field and a list. The list is server data. You e/for-by over it on the server and stream the diffs to the client. Now the client can do fine-grained updates to the list. You see this pattern all the time
(e/server
(e/for-by key-fn [v (query)]
(e/client
(dom/div ...))))
What happens when you type in a new todo item? .... Hmm, that's client data. The value goes to the server over the wire. The server transacts it to the db. The db acks the write. The server e/for-by streams the diff over. Now the client can finally render your new item! But ugh, the latency!
What can we do? Ideally we'd like to optimistically render the in-flight items in the same e/for-by list. But it's on the server!
So what semantics are we looking for exactly? Well, we want to optimistically include a client value in a server collection and when the server responds with success/failure the value stays or gets removed or gets a red X with a retry button or ... Up to the user. The information should be available for you to decide.
So, can we add a client value to a server collection? No..
Differential electric to the rescue! If we operate on diffs, we can fold over them in any way we like. We can take client and server diffs and fold them into a single e/for-by-like loop. This is what we mean when we say e/for-by is not bound to a single peer anymore.
Of course this solves many more use cases and opens up many new possibilities.
Hopefully @U053XQP4S will correct any wild inaccuracies 😉#2023-09-0714:56telekidThanks for the explanations! really cool#2023-09-0715:04telekidtook a few readings, but now makes perfect sense and seems almost obvious in hindsight#2023-09-0715:10telekidit's funny, I bet everyone working on, say, a REST API has at some point had the realization that it is much easier to design all of your queries to return collections even if you happen to be building a user endpoint rather than a users endpoint. It makes perfect sense to me to lift that all the way back to the expression level.#2023-09-0715:12telekid(obviously the motivations are different)#2023-09-0715:14telekidI do wonder what electric's learning curve will look like eventually. "if you think you had to unlearn things when you started writing Clojure, wait until you try electric..."#2023-09-0715:15telekidI guess that will work itself out though#2023-09-0716:01Dustin Getzthe differential stuff is internals from the beginner's perspective, e/for-by should look the same as it does today, it's the implementation of it that changes. The big idea is that today e/for-by is a userland macro and that macro does diffing in userland; after differential electric, this logic is absorbed into the electric runtime and e/for-by is just syntax sugar over the amb multiplexing stuff Leo was talking about. The advantage of having the electric runtime own the diffing is that the network planner can be aware of it and take advantage of it.#2023-09-0716:03Dustin GetzWhen you use a streaming datasource like Materialized, you don't even want e/for-by, you want "the second stage of it" (the reduction over diffs), not the diffing part since Materialized gives us pre-diffed output. So differential electric also unbundles this better.#2023-09-0707:29weiam I using this wrong?
(e/client
(if (exists? js/jQuery)
(dom/div (dom/text "jquery"))
(dom/div (dom/text "no jquery"))))
results in an error: jQuery is not defined
I believe this works without errors in vanilla cljs:
(if (exists? js/jQuery)
(println "jQuery"))
(println "no jQuery"))#2023-09-0707:33weioh this seems to work:
(exists? (aget js/window "jQuery"))
#2023-09-0707:41xificurCon what electric version are you?#2023-09-0708:09weicom.hyperfiddle/electric {:mvn/version "v2-alpha-428-g22937f75"}#2023-09-0708:17JHi guys! With the latest version I have sometimes a StackOverFlow error on the compile time. I comment/uncomment some part and the app compile again. Have you got this kind of behaviour?#2023-09-0708:47xificurCwe've encountered cljs build failures due to exceeding the default stack size, e.g. in the examples app. You can https://github.com/hyperfiddle/electric-examples-app/commit/dab45788afc5f1c8128800f3c7e9ba2b869120f7#2023-09-0708:48JEven in dev mode?#2023-09-0708:50xificurCI don't think I've seen it in dev mode, although I guess it's bound to happen at some point. Unless you have a real stack overflow? You can try increasing the stack size and see#2023-09-0710:29JIt’s weird the app is very light.#2023-09-0710:37Dustin Getzplease post the error #2023-09-0710:45JHere the stack trace on dev compile time start. I try to create a gist with the code.
--------------------------------------------------------------------------------
5 | [quizy.web-ui.core :as core]))
6 |
7 | (def electric-main
8 | (e/boot
---------^----------------------------------------------------------------------
null
StackOverflowError:
java.lang.ClassLoader.defineClass1 (ClassLoader.java:-2)
java.lang.ClassLoader.defineClass (ClassLoader.java:1017)
java.security.SecureClassLoader.defineClass (SecureClassLoader.java:174)
jdk.internal.loader.BuiltinClassLoader.defineClass (BuiltinClassLoader.java:800)
jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull (BuiltinClassLoader.java:698)
jdk.internal.loader.BuiltinClassLoader.loadClassOrNull (BuiltinClassLoader.java:621)
jdk.internal.loader.BuiltinClassLoader.loadClass (BuiltinClassLoader.java:579)
jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass (ClassLoaders.java:178)
java.lang.ClassLoader.loadClass (ClassLoader.java:522)
clojure.core/elide-top-frames (core.clj:4803)
clojure.core/ex-info (core.clj:4807)
clojure.core/ex-info (core.clj:4807)
cljs.analyzer/error (analyzer.cljc:780)
cljs.analyzer/error (analyzer.cljc:776)
cljs.analyzer/analyze (analyzer.cljc:4366)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4362)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4360)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze-const (analyzer.cljc:2539)
cljs.analyzer/analyze-const (analyzer.cljc:2536)
cljs.analyzer/fn--3047 (analyzer.cljc:2551)
cljs.analyzer/fn--3047 (analyzer.cljc:2547)
clojure.lang.MultiFn.invoke (MultiFn.java:252)
cljs.analyzer/analyze-seq* (analyzer.cljc:4080)
cljs.analyzer/analyze-seq* (analyzer.cljc:4078)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4085)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4083)
cljs.analyzer/analyze-seq (analyzer.cljc:4109)
cljs.analyzer/analyze-seq (analyzer.cljc:4087)
cljs.analyzer/analyze-form (analyzer.cljc:4296)
cljs.analyzer/analyze-form (analyzer.cljc:4293)
cljs.analyzer/analyze* (analyzer.cljc:4349)
cljs.analyzer/analyze* (analyzer.cljc:4341)
cljs.analyzer/analyze (analyzer.cljc:4369)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4362)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4360)
cljs.analyzer/analyze (analyzer.cljc:4352)
shadow.build.cljs-hacks/shadow-parse-invoke*/fn--5892/fn--5893 (cljs_hacks.cljc:828)
clojure.core/mapv/fn--8535 (core.clj:6979)
clojure.core.protocols/fn--8249 (protocols.clj:168)
clojure.core.protocols/fn--8249 (protocols.clj:124)
clojure.core.protocols/fn--8204/G--8199--8213 (protocols.clj:19)
clojure.core.protocols/seq-reduce (protocols.clj:31)
clojure.core.protocols/fn--8234 (protocols.clj:75)
clojure.core.protocols/fn--8234 (protocols.clj:75)
clojure.core.protocols/fn--8178/G--8173--8191 (protocols.clj:13)
clojure.core/reduce (core.clj:6886)
clojure.core/mapv (core.clj:6970)
clojure.core/mapv (core.clj:6970)
shadow.build.cljs-hacks/shadow-parse-invoke*/fn--5892 (cljs_hacks.cljc:828)
clojure.lang.Delay.deref (Delay.java:42)
clojure.core/deref (core.clj:2337)
clojure.core/deref (core.clj:2323)
shadow.build.cljs-hacks/shadow-parse-invoke* (cljs_hacks.cljc:960)
shadow.build.cljs-hacks/shadow-parse-invoke* (cljs_hacks.cljc:817)
cljs.analyzer/parse-invoke (analyzer.cljc:3836)
cljs.analyzer/parse-invoke (analyzer.cljc:3834)
cljs.analyzer/analyze-seq* (analyzer.cljc:4081)
cljs.analyzer/analyze-seq* (analyzer.cljc:4078)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4085)
cljs.analyzer/analyze-seq*-wrap (analyzer.cljc:4083)
cljs.analyzer/analyze-seq (analyzer.cljc:4109)
cljs.analyzer/analyze-seq (analyzer.cljc:4087)
cljs.analyzer/analyze-form (analyzer.cljc:4296)
cljs.analyzer/analyze-form (analyzer.cljc:4293)
cljs.analyzer/analyze* (analyzer.cljc:4349)
cljs.analyzer/analyze* (analyzer.cljc:4341)
cljs.analyzer/analyze (analyzer.cljc:4369)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4362)
cljs.analyzer/analyze (analyzer.cljc:4352)
cljs.analyzer/analyze (analyzer.cljc:4360)
cljs.analyzer/analyze (analyzer.cljc:4352)
shadow.build.cljs-hacks/shadow-parse-invoke*/fn--5892/fn--5893 (cljs_hacks.cljc:828)
clojure.core/mapv/fn--8535 (core.clj:6979)
clojure.lang.PersistentList.reduce (PersistentList.java:141)
#2023-09-0710:52JThe code of the electric app: https://gist.github.com/jeans11/3f5df7d2b134e611167705322bdadd13#2023-09-0711:01Dustin Getzplease add deps.edn as well#2023-09-0711:24xificurCI suspect some typo masquerading as this. You should comment bisect* to find the form causing it.
*comment out half of your code, see if error persists, recurse#2023-09-0711:35Dustin GetzIf you can send us a repo to clone where we can reproduce it, we can improve the error (if this is a syntax error) or otherwise make this more pleasant#2023-09-0712:50JI send you an invite to the repo#2023-09-0809:11JWhen I remove the .shadow-cljs cache, start a repl and the app, everything works! Made some change on the app, etc... no errors. When I close / start my repl and start the app, I got the StackOverFlow error.#2023-09-0708:45joshcholow-effort question, would there be any issues scaling with electric? off the top of my head i don't see anything beyond the normal backend level concerns#2023-09-0708:49xificurCno, your backend should scale just fine#2023-09-0710:31JI have this warning with the hyperfiddle.history namespace. Any advices?#2023-09-0710:36Dustin Getzwhat electric version? please make sure you upgrade to 428#2023-09-0710:47JYep I’m on the 428#2023-09-0710:48Jv2-alpha-428-g22937f75#2023-09-0711:18xificurCif you remove the shadow cache do the warnings persist? (`rm -rf .shadow-cljs`). You'll need to start the app twice to be sure the warnings are gone, first run fills the cache, second uses it#2023-09-0714:41henrikHello World from Rama+Electric#2023-09-0714:44henrikI.e., just WordCount with UI.#2023-09-0715:55Dustin Getzcode?#2023-09-0716:07henrikI’m putting together a gist with the salient parts. I’ve got to remove some Multiplyisms.#2023-09-0716:33henrikhttps://gist.github.com/eneroth/bb22966fa76f32caae04200ca21f6332#2023-09-0716:36henrikThat should be pretty much it.
@U09K620SG Interesting interaction between Electric and Rama: In subscribe, I had to use proxyAsync over proxy, or pretty much everything locked up. Weirdly, the first mounted sub would be OK, but the second would block indefinitely. Luckily, this issue didn’t arise with proxyAsync.#2023-09-0716:52henrikA general note: A global partition wouldn’t be a good idea if you want 100 million bots counting words. A Rama-maintained toplist would be the way to go.#2023-09-0718:38weithanks @U06B8J0AJ! trying out your example, I'm getting an error, "The required namespace "com.rpl.rama.Depot" is not available". are you using these versions of rama?
com.rpl/rama {:mvn/version "0.9.4"}
com.rpl/rama-helpers {:mvn/version "0.9.0"}#2023-09-0718:41henrik@U066TMAKS Yep, those are the ones. It’s only Depot that’s the problem, nothing else?#2023-09-0718:44weiah nevermind, i was messing up clj/cljs. i realize now your example is split into several files#2023-09-0718:46weiwhat goes into #_get_cluster_here? guess i need to read up on rama more#2023-09-0718:51weitrying this:
(e/def cluster (e/server (.create InProcessCluster)))
#2023-09-0718:52henrikAh yeah, sorry, I didn’t include that because it’s delivered by our boot system. I’ll update the gist with some general Rama init stuff.#2023-09-0718:52henrik(InProcessCluster/create)
#2023-09-0718:53henrikBut you shouldn’t put it in the e/def, you should have it globally somewhere.#2023-09-0718:53weiright#2023-09-0718:54Dustin Getzregarding the lockup, .proxy and .proxyAsync are rama methods, are they documented? i have no idea what they do
(I know superficially they are emitting diffs but i dont know if they are doing e.g. thread stuff)#2023-09-0718:57henrik@U066TMAKS Try this:
;; Change require to:
[app.modules.wordCount :refer [->SimpleWordCountModule]]
;; Replace the `e/def` with
#?(:clj (defonce cluster
(let [c (InProcessCluster/create)]
(.launchModule c (->SimpleWordCountModule) (LaunchConfig. 1 1))
c)))#2023-09-0718:59henrikSorry, a few edits above, it was hastily written.#2023-09-0719:01henrik@U09K620SG https://redplanetlabs.com/javadoc/com/rpl/rama/PState.html#proxy-java.lang.Object-com.rpl.rama.Path- and below#2023-09-0719:01henrikAnd https://redplanetlabs.com/docs/~/pstates.html#_reactive_queries#2023-09-0719:02Dustin Getz> Blocks until initial query finishes#2023-09-0719:04Dustin Getzi don't fully understand what that means, i'll have to look more closely when i have more time#2023-09-0719:06henrikI think it means “blocks until it has an initial value to return”, since you can run .get and stuff on the proxy object.
Anyway, it’s not urgent, merely interesting. proxyAsync is fine, since the return value isn’t needed until Electric does cleanup.#2023-09-0719:06Dustin Getzim kinda shocked that registering a callback on something is a blocking operation ever#2023-09-0719:07henrikIt does two things: runs a query and registers a callback. Maybe those should have been two separate operations, but here we are.#2023-09-0719:11henrikI read the semantics like, imagine you could run (atom), and deref would block until the atom receives the first value. #2023-09-0719:15henrik@U066TMAKS I updated the gist with what I think would work for as a start. Let me know if it doesn’t.#2023-09-0719:17weithanks, had to step out for a sec but i will review shortly#2023-09-0720:18weihttps://github.com/yayitswei/rama-electric-example
currently seeing errors in startup, and then java.net.BindException: Address already in use if i try to eval the defonce cluster line. i suspect it's a sequencing issue, unless there's some port conflict that i'm not aware of#2023-09-0720:42henrikYeah, it’ll need to be managed. You can .close the cluster if you need to reevaluate it, or otherwise use some system state lib like Integrant.#2023-09-0721:14henrik@U066TMAKS Sent you a pull request#2023-09-0722:27weihappy to say we got this running. needed to match rama's internal version of clojure, 1.11.1 (see updated example). thanks @U06B8J0AJ for your patience troubleshooting!#2023-09-0716:59Sagar VrajalalI'm trying to setup some basic routing in my app using missionary and reitit, and took some inspiration from this piece of code i found on this channel
https://github.com/lumberdev/tesserae/blob/cea33f19b46892abb78feb99d51af2dd54849435/src/tesserae/ui/app.cljs
As soon as I navigate to another route from a component (using href), I get an error in my console:
TypeError: Cannot read properties of undefined (reading 'insertBefore')
and the Router div gets emptied out
If I comment out the Footer from my root component, the routing works as expected and the error is gone. I'm fairly new to frontend development so I am a bit stumped here, I think I must be doing something silly
Here is a snippet of my code:
https://gist.github.com/S4G4R/fbfbebde5a8b42d1b0050ad126f1a348#2023-09-0721:37Dustin Getza few comments#2023-09-0721:40Dustin Getz• i think cljs reader conditional doesn’t do what you want it to in e/defn, not sure if that is impacting this
• with-reagent may need adjustments, it was provided as an example not as a robust component
• what is Footer? the gist you provided is incomplete#2023-09-0721:44Dustin Getzi currently suspect the reader conditional (without seeing the full context). The Footer form is being elided entirely, causing an arity error which JS interprets as passing undefined. Fix of this would be to not use a reader conditional. Presumably you inserted a reader conditional here because the linter told you to. Unfortunately linters give broken results with Electric, you have to ignore them or turn them off until we figure out how to fix them#2023-09-0806:33Sagar VrajalalApologies, I added the Header and Footer to the gist.
I indeed added the reader conditionals because the linter was complaining, unfortunately getting rid of them didn't help either.#2023-09-0806:34Sagar VrajalalHere's the full error message I see in the console:
Cannot read properties of undefined (reading 'insertBefore')
in reactive (fn [] ...)
in reactive (fn Hello [] ...)
in (case :hello ...) in com/sagar/casa/ui/router.cljc line 38
in reactive (defn Router [] ...) in com/sagar/casa/ui/router.cljc line 34
in reactive (fn [] ...)
in reactive (defn Root [] ...) in com/sagar/casa/ui.cljc line 11
in (try ...)
TypeError: Cannot read properties of undefined (reading 'insertBefore')
at $sagar$casa$ui$reagent$react_root_hook__2 [as cljs$core$IFn$_invoke$arity$2] (reagent.cljc:24:44)
at eval (runtime.cljc:641:16)
at eval (runtime.cljc:601:13)
at eval (Continuous.cljs:144:8)
at missionary$impl$Continuous$suspend (Continuous.cljs:144:7)
at Object.eval [as missionary$impl$Fiber$Fiber$swich$arity$2] (Continuous.cljs:16:23)
at Object.missionary$core$switch [as switch$] (core.cljc:176:17)
at hyperfiddle$electric$impl$runtime$variable_$_cr62862_block_3 (runtime.cljc:619:9)
at eval (impl.cljc:60:19)
at Function.G__61998__0 [as cljs$core$IFn$_invoke$arity$0] (impl.cljc:60:35)#2023-09-0807:31xificurCcan I see with-reagent and Hello?#2023-09-0809:19Sagar VrajalalI was able to fix this. I had an older copy of with-reagent, this change fixed it
https://github.com/hyperfiddle/electric/commit/dbbd147a0dc6748d5bf420c697d9d2221409cbfa
Thank you for prompting me to look at with-reagent again @U09FL65DK 😅#2023-09-0804:27Jordan CalderwoodHow do I make reusable components like:
(e/defn Card
[something body]
…stuff
(dom/div body))
I don’t quite understand how to delay the evaluation of the body, it always renders first in the dom before “stuff”#2023-09-0806:08henrikThe only way I’ve found to do this currently is with macros.#2023-09-0807:18henrikE.g.,
(defmacro button
[{:keys [kind]
:or {kind :white}
:as props}
& children]
`(dom/button …
~@children))#2023-09-0807:33xificurCcan you give an example of a component that doesn't behave as you expect it to? (do (dom/div (dom/text "a")) (dom/div (dom/text "b"))) renders in the order a then b#2023-09-0807:35xificurCah, do you mean you call it like (Card. something (dom/div (dom/text "c")))? The simplest is to thunk it, i.e. send in (e/fn [] (dom/div)) and call it (dom/div (Body.))#2023-09-0807:37henrikYeah, this is more about composition and reusability as I understand it. Like:
(Container.
(Section.
(Item.)
(Item.)
(Item.))
(Section.
(Thing.)
(Thing.)
(Thing.)))
Where e.g. Section is some bundle of structure and behaviour that is agnostic to its children.#2023-09-0807:39henrikI attempted (unsuccessfully) to write a macro to make this generic early on.#2023-09-0807:40xificurCI think this is people looking at it with the reagent glasses on. When writing clojure code, it's obvious to everyone in (section (item) (item) (item)) items will be evaluated before section gets called#2023-09-0807:42henrikIt’s not about obviousness, it’s about getting at the pattern of reusable “components”. I understand why Item is evaluated before Section.
To be fair, it’s not about Reagent specifically. It goes back to HTML, and virtually every frontend framework mimics that structure. I.e., it’s possible to place an on-screen element inside of another on-screen element, where the second one doesn’t necessarily know or care about what you place inside it.#2023-09-0807:44henrikElectric, of course, is a different model, so it’s obvious why this doesn’t happen.
I don’t think it’ll be obvious to every person that tries out Electric, and I do think it might put some constraints on what type of higher order “components” are possible for a layman to write, and therefore might impact what the ecosystem around Electric ends up looking like.#2023-09-0807:54xificurC> It goes back to HTML, and virtually every frontend framework mimics that structure. I.e., it’s possible to place an on-screen element inside of another on-screen element, where the second one doesn’t necessarily know or care about what you place inside it.
As I showed this is true for electric as well. (Section. Item Item Item) sends in 3 components and Section will call new on them, not caring about their structure#2023-09-0807:57xificurCit's true we're using novelty budget, so users need adapting. But there's nothing stopping one from composing and building higher-order components#2023-09-0807:59henrikRight, but say Item takes arguments. And say it’s not one level, but 3-5 levels of children. It’s possible, I’m just saying that it’s not necessarily obvious how to do it, and it’s not necessarily ergonomic.#2023-09-0808:02xificurCif you encounter a non-obvious case, feel free to post a snippet and we'll help#2023-09-0808:04henrikI think it’s possible to extrapolate from the above, no? Hang on, I’ll provide another example.#2023-09-0808:10henrikPseudocode meant to mirror something you might see in React etc.:
(Menu. {:orientation :vertical}
(Section. {:title "Profile"}
(Item. {:title "Edit profile"
:image (Image. (UserImage.))}
(dom/on "click" …))
(Item {:title "Log out"}
(dom/on "click" …)))
(Section. {:title "Documents"}
(for-by [document documents]
(File. {:icon (DocumentIcon.)} document))))
You can get to this with macros or lambdas or a combination thereof. You can also forgo the reusable part and make ProfileSection and DocumentSection, ProfileItem and LogOutItem etc., and not compose them this way.#2023-09-0808:12henrikAlso, if you believe it to be a non-issue, I won’t press it. For me personally, it’s not a show-stopper.#2023-09-0808:14henrikI’m just trying to shed some light on what I believe led to the original question.#2023-09-0808:51xificurCsure, thanks! There are many ways to structure your example, some brainstormed ideas
• if Menu is just dom/menu with some styles you could extract the styles in a separate helper and write (dom/menu (MenuStyles. {:orientation :vertical}) ...)
• if you want to mimic the dom helpers you'd write (demacro menu [styles & body] (dom/menu (MenuStyles. ~styles) ~@body))`
• you can also thunk all the way, although in this example it would get pretty verbose
• the dom/on examples cannot work as-is, you really need a macro for that. Also, you're injecting behavior, which means the component isn't really a black box, suggesting the abstraction is leaking
• these high-level components tend to turn into god-class-like behemoths, maybe one should just inline them and factor out the commonalities into helper functions
The list resembles typical coding questions - where to put abstractions in and where to leave them out, how to factor our code, do we want to write a macro for syntax sugar etc.#2023-09-0808:59henrikRegarding god-class stuff, the goal is the opposite: rather than having a huge props API to some enormous thing that internally hides/shows components, flips behaviour on it’s internally constructed children, etc. in response to arguments, you try to shrink the API and expose the generic capability of being able to take children from the outside. The constraint being that the parent can’t then inject behaviour on them—this is done from the outside at the moment of composition.
There’s no silver bullet, and it usually ends up being a trade-off where the practical solution is some level of props API + semi-generic ability to take some class of children. A menu component can’t take arbitrary children, obviously, only those that make sense in a menu. But it can be agnostic to the children of the children, etc., which a mega-component that controls all levels of children from the top down, internally, can’t.#2023-09-0810:41Dustin Getzthe unit of reuse is lambda, components are lambda, this appears to be a discussion of syntax , not of reuse#2023-09-0810:48henrikYeah, it’s mostly about ergonomics and approachability. Stuff from hyperfiddle.electric-dom2 compose declaratively, as does the stuff from ui, whereas your own “components” do not. This is surprising, hence the original question.#2023-09-0810:53henrikTo be absolutely clear, my points are not:
• About bashing Electric: it’s fantastic
• Saying that Electric’s model is wrong
• Making any arguments for how you should prioritize or structure your time
It’s only about trying to convey that for some, this is surprising. I then go on to speculate that this is one variable that might impact how easily libraries, third-party components etc. made for Electric proliferates. I think an “at hand” pattern for dealing with it would help. With which you might disagree.#2023-09-0811:11Dustin GetzAh ok specifically the change in apparent semantics between the dom macros and regular Electric code is what is surprising, that i accept as true#2023-09-0811:13Dustin GetzOur official stance on this is that we care much more about language semantics than having easy dom authoring syntax this year. Committing to a syntax too soon (before understanding advanced use cases) risks getting it wrong#2023-09-0811:13Dustin Getzsee https://github.com/hyperfiddle/electric/issues/32#2023-09-0811:14Dustin Getzas for data composition vs lambda composition, it's important to note that lambda composition is more fundamental, data can be implemented in terms of lambda#2023-09-0811:17Dustin Getz{:a {:b 2}} vs (hash-map :a (hash-map :b 2))#2023-09-0811:26Dustin Getzyou could imagine an OOP pattern like this quick sketch, but it has issues to work out
(e/defn Div [& children]
(e/fn []
(let [el (goog.dom/createElement "div")]
(.appendChild dom/node el)
(binding [dom/node el]
(e/for [Child children] ; dynamic traversal at runtime
(Child.))))))
(Div. (Div.) (Div. (Div.)))
#2023-09-0811:28Dustin Getzanyway the upcoming differential Electric has implications in how lower level things are done, which is another reason why we choose not to work on syntax until Electric is more complete than it is today#2023-09-0811:37Dustin Getz@U02ABMYDCGK the intuition to use here is that in an effectful dom abstraction, dom effects (say dom/text ) evaluate like println , does that match what you experience?#2023-09-0813:46Jordan CalderwoodYes. That matches what I am seeing and works nicely. I built a nice little UI but I have a lot of repeated code.
I am just looking for a way to pass children into a component of my creation. Specifically a Card component with an x in the top right that takes a call-back to reset an atom and arbitrary children that will render inside the card. The effectful nature of electric dom makes that hard (for me!) to do. I don’t want to write macros for components 😝 #2023-09-0813:47Jordan CalderwoodAlso, thanks for building this thing. It is a ton of fun and feels magical. #2023-09-0815:58Dustin GetzDo you have your answer? wrap your dom markup in e/fn and pass it around as lambda, call it at the place where you want to splice it in#2023-09-0816:09Jordan CalderwoodAh I understand now what y’all were getting at about lambda as the abstraction. Works for me! Thanks for the help. #2023-09-0816:10Jordan CalderwoodOne other thing that “got me” was having to call new on the e/fn lambda. #2023-09-0809:15henrikShower thought: Clojure has clojurians, so it follows that Electric has… Electricians?#2023-09-0809:52tatutthe end users of electric apps are then an electrified audience?#2023-09-0811:10Dustin Getzha ha#2023-09-0812:30mattiasIf I have the following let block
[!a (atom {:k1 (some-fn) :k2 val2})
a (e/watch a)]
and val2 is some dynamic value (e.g. derived from path , defined with e/def like in https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc#L63C50-L63C57).
1. what happens when val2 changes (in this case, due to navigating to a new page)?
a. Do we get a new instance of !a or is the :k2 value of !a updated?
b. If we get a new instance of !a, is some-fn also called again when val2 changes?
2. if we (swap! !a assoc :k2 123) , and then val2 gets a new value (by navigating to a new page), will (get a :k2) have that new value or 123 ?#2023-09-0812:44xificurC1a: yes, you get a new value. A clj(s) function is re-run when any of its arguments change. In this case the hash-map is re-built (the hash-map literal compiles down to (hash-map :k1 (some-fn) :k2 val2)), so atom re-runs as well#2023-09-0812:44xificurC1b: no, some-fn is cached, since no arguments changed to the function call#2023-09-0812:46xificurC2: yes, you'll see the new value, since the change of val2 builds a new atom which gets a new watch etc#2023-09-0812:47xificurCwhat is the purpose of pushing a reactive value into an atom to later watch and retrieve it again?#2023-09-0812:50mattiasI kind of stumbled on this by trying something silly (i rewrote it, because it felt wrong). But since it worked, I started wondering why it works and what happens under the hood. Good chance to get a better intuition for Electric.
Thanks for the explanations, all clear!#2023-09-0812:53xificurCno problem! In cases where you have to do this it might be simpler to pull the reactive part out
(let [!a (atom {:k1 (some-fn)})]
(swap! !a assoc :k2 val2)
(e/watch !a))
Now the atom is stable and only the swap! re-runs#2023-09-0813:04mattiasOh, cool, I was looking for sth like that!
What about a slightly different scenario:
(let [!a (atom {:k1 1
:k2 2})
a (e/watch a)
k1 (a :k1)
k2 (a :k2)]
(Component1. k1)
(Component2. k2)
(Component3. a))
If (a :k1) changes, do both Component1 and Component3 get re-rendered? Or only parts of them which actually use the k1 value (either the k1 let binding directly, which is passed down, or e.g. via (get a :k1) somewhere downstream)?#2023-09-0813:09mattiasBased on what you just said, seems like both [k1 (a :k1)] and [k2 (a :k2)] would be re-evaluated, because their input changed (I'm assuming (a :k1) is the same as (get a :k1) . And then components 1 and 3 would be re-computed, since their arguments changed, but component 2 would not since its input value didn't change?#2023-09-0813:09xificurCthe update propagates through them and only calls that depend on the value will re-run. It's a DAG#2023-09-0813:11xificurCcorrect, (:k2 a) will re-run but return the same value, therefore downstream dependencies will use the cache (last run's result)#2023-09-0813:13mattiasOK, so the caching is really granular and done at the level of map values as well, rather than at the whole atom level. Really impressive 🔥#2023-09-0812:43joshcho2 questions: is there any reason to consider sth like htmx with electric? also, what would it take to use electric for TUI?#2023-09-0812:48xificurChtmx is built around RESTful communication. Electric abstracts the network away. I don't see a good fit here personally#2023-09-0812:51xificurCre TUI - definitely doable, one would need to build the reactive layer like we built for the DOM, <https://github.com/hyperfiddle/electric/blob/983544cb84819ab1ff3bfefe316192da648bd18a/src/hyperfiddle/electric_dom2.cljc%7Cwhich is <400loc> with some tests and deprecated fns#2023-09-0813:02joshchosounds like a fun project#2023-09-0813:03joshchoelectric as shared communication layer would be quite amazing#2023-09-0813:16joshchowhat library would u recommend if i want to do some dom manipulations#2023-09-0813:24xificurCwhat manipulations are you thinking of?#2023-09-0813:28joshchonow that i think about it, not much. declarative ui takes care of most things. but i was considering sth like https://github.com/plumatic/dommy for making handlers in vanilla js less painful to write#2023-09-0816:12Dustin Getzare you asking for a nicer electric-dom authoring syntax? #2023-09-0823:05joshchoyeah, kinda. Just manipulating the dom in event handlers#2023-09-0823:05joshchoVanilla javascript functions are not ergonomic#2023-09-0818:57nivekuilis there a demo of typeahead/tag-picker somewhere? need to make a "start typing, show existing items, if none of those work then make new thing" interface#2023-09-0819:06Dustin GetzWe have several different impls spread across projects that need consolidation#2023-09-0819:06Dustin GetzOne that is bundled with electric is this, it's old though – https://electric-demo.fly.dev/(wip.tag-picker!TagPicker)#2023-09-0819:06Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src-docs/wip/tag_picker.cljc#2023-09-0819:12nivekuilhmm, getting maximum call stack size exceeded if I use tag-picker within a (for-by (dom/div ...#2023-09-0819:13nivekuilthis works
(e/for-by
:id [{:keys [id text editing? importance base-importance]}
(new (q '[[id ::node-type :problem]
[id ::node-text text]
[id ::editing? editing?]
[id ::importance importance]
[id ::base-importance base-importance]
#_[id ::blocks blocks]]))]
(let [x (atom nil)]
(ui/tag-picker (e/watch x)
(e/fn [add] nil)
(e/fn [disj] nil)
(e/fn [search-input] nil)
(e/fn [e] nil)))
...#2023-09-0819:13nivekuilthis doesnt
(e/for-by
:id [{:keys [id text editing? importance base-importance]}
(new (q '[[id ::node-type :problem]
[id ::node-text text]
[id ::editing? editing?]
[id ::importance importance]
[id ::base-importance base-importance]
#_[id ::blocks blocks]]))]
(dom/div
(let [x (atom nil)]
(ui/tag-picker (e/watch x)
(e/fn [add] nil)
(e/fn [disj] nil)
(e/fn [search-input] nil)
(e/fn [e] nil))) #2023-09-0819:14nivekuilthat extra layer of nesting is breaking something I guess#2023-09-0901:21nivekuilis there a way to force remount a component?#2023-09-0911:08Dustin Getzwe don’t want you doing that, yes there is a way#2023-09-1206:59joshchoHow does one go about debugging something like this? I think I face something like this couple times a day, and I have no idea where to look.
------ ERROR -------------------------------------------------------------------
File: /home/pollock/wakmantle/src/user.cljs:8:3
--------------------------------------------------------------------------------
5 | hyperfiddle.electric-dom2))
6 |
7 | (def electric-main
8 | (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
null
StackOverflowError:
clojure.lang.RT.boundedLength (RT.java:1790)
clojure.core/apply (core.clj:669)
clojure.core/update-in/up--6922 (core.clj:6220)
clojure.core/update-in (core.clj:6221)
clojure.core/update-in (core.clj:6207)
cljs.analyzer/register-constant!/fn--2398 (analyzer.cljc:537)
clojure.lang.Atom.swap (Atom.java:37)
clojure.core/swap! (core.clj:2369)
clojure.core/swap! (core.clj:2362)
#2023-09-1207:02xificurCyeah, the error isn't very helpful 🙂 Is this on hot code reload? Sometimes restarting the REPL produces a cleaner error (or makes it go away)#2023-09-1207:02xificurCotherwise I'd just comment bisect#2023-09-1207:17caleb.macdonaldblackProbably related to this: https://clojurians.slack.com/archives/C7Q9GSHFV/p1693933276642909#2023-09-1207:18caleb.macdonaldblackand this: https://clojurians.slack.com/archives/C7Q9GSHFV/p1694074633693519#2023-09-1207:19caleb.macdonaldblackWhen I experienced RangeError: Maximum call stack size exceeded in the browser. I also had shadow failing to compile with StackOverflowError at the same time as I was playing around trying to work out what was going on.#2023-09-1207:23JI solve the problem with the -Xss option in the jvm-opts into my dev alias.#2023-09-1207:24JLike this:#2023-09-1207:24caleb.macdonaldblackah#2023-09-1207:24caleb.macdonaldblackhttps://github.com/hyperfiddle/electric-starter-app/blob/2033b010393237c0b87c59c981fdd31155c05e3e/deps.edn#L18#2023-09-1210:18Dustin Getzcompile time stackoverflow usually just means just increase comptime stack size (available to shadow), electric compilation requires more than 1MB stack#2023-09-1211:06joshchoYeah, I was actually just running out of memory I think, needed -Xss8m. Was using some large literals instead of an actual db for now.#2023-09-1212:31joshchoI sat down for 16 hours or sth and built a full electric app (https://semantle.com/ clone) in 800 LOC. Pretty weird to keep all the moving components in my head and "not lose any context" while doing it. The https://pages.cs.wisc.edu/~remzi/Naur.pdf never has to be reconstructed. Possible because of 1 file and less LOC. Also been accruing good macros along the way.#2023-09-1213:02xificurCare you planning to share the code?#2023-09-1214:37joshchohttps://github.com/joshcho/wakmantle, FYI it's in Korean tho and for a streamer fanbase#2023-09-1214:29grounded_sageI know this is not the intention with Electric. But does it make sense and is it possible to easily decouple it from backend. I know people complain about the authoring experience but I actually kind of like it haha. I would build frontends entirely in electric dom functions. Actually looking at building a copy paste UI kit for electric and it would be super nice to be able to host it statically.#2023-09-1214:46danielstocktonAlong the same lines, I have wondered how one might provide an API to users when building an app in electric. I guess there needs to be some kind of formal separation between the electric backend and main API. The electric backend would be a thin proxy.#2023-09-1215:44xificurC@U05095F2K I parsed 2 questions:
1. can one write client-only code with electric?
2. can electric generate a static website?
For 1 - you can write client-only electric code today. You still need the machinery (a server with a websocket to connect to) but there should be no other traffic than the initial connect. This is not a priority for us though, our goal is to enable building rich dynamic UIs
For 2 - this is a common request that is out of scope for now. Electric is a reactive language, you'll be better off using a static site generator#2023-09-1215:47xificurC@U0DHDNZR9 since electric builds on top of clojure(script) you can use the same tools you're used to. The UI can (and should!) communicate with the server directly, but if a third party needs to request data from the backend you can build e.g. an HTTP API alongside as usual#2023-09-1215:52danielstocktonYep, i was talking more about ways to avoid repeating yourself between electric vackend code and code to serve an api alongside. I thought @U05095F2K was getting at something similar i.e. hiw to write reusable components that fetch data from unknown backend#2023-09-1215:56xificurC> i was talking more about ways to avoid repeating yourself between electric vackend code and code to serve an api alongside
typically the business logic is decoupled from the controller, right? If you have fetch-user-info the controller will call it and in electric you'd call it directly, without an HTTP endpoint#2023-09-1216:00xificurC> how to write reusable components that fetch data from unknown backend
How does an HTTP endpoint guard you from unknown backends? If you GET /fetch-user-info/11 any backend you connect to needs to provide that endpoint. In electric for (e/server (user-info/fetch-info 11)) you need a namespace aliased to user-info that has a function called fetch-info.
The other option is to take data as arguments, in which case you'll have a data shape dependency#2023-09-1216:10danielstocktonYou can supply an endpoint but yes you have a data shape dependency. Not saying that http guards you against unknown backends, but you can pick and choose which backend to use.#2023-09-1216:11danielstocktonIn electric you would have an implicit contract to implement a certain function i.e. the same data shape dependency.#2023-09-1216:33grounded_sageOkay I guess it can be hosted on free tier of a service or people can spin up themselves to see it.#2023-09-1216:40grounded_sageI'm basically looking to replicate https://ui.shadcn.com/#2023-09-1216:40grounded_sageWhen I pass in something other than a string for Collapsible it doesn't render the button. Is what I am writing expected to be valid?
(e/defn Collapsible [{:keys [trigger content disabled? default-open?]}]
(let [!open? (atom default-open? true false)
open? (e/watch !open?)]
(dom/button
(dom/props {:type "button"
:aria-expanded open?
:disabled disabled?
:data-state open?
:data-disabled disabled?})
(dom/on "click" (e/fn [_] (reset! !open? (not @!open?))))
(if (string? trigger)
(dom/text trigger)
trigger))
(dom/div
(dom/props {:hidden (not open?)})
(if (string? content)
(dom/p (dom/text content))
(content)))))
(dom/div
(dom/h1 (dom/text "Collapsible"))
(dom/p (dom/text "An interactive component which expands/collapses a panel."))
(dom/div (dom/props {:class "flex items-center justify-between space-x-4 rounded-md p-4 border"})
(dom/h4
(dom/props {:class "text-sm font-semibold"})
(dom/text "@peduarte starred 3 repositories"))
(Collapsible. {:trigger (dom/div
(dom/img (dom/props {:src "./radix-icons/chevron-up.svg"}))
(dom/img (dom/props {:src "./radix-icons/chevron-down.svg"})))
:content "Yes. Free to use for personal and commercial projects. No attribution
required."})))#2023-09-1218:41grounded_sagethought I would do something simple to take a break from hard stuff but I somehow feel this moves into macro territory??#2023-09-1218:45xificurCthe function you pasted takes a map as its argument, what do you mean you're passing in a string?#2023-09-1218:46xificurC(reset! !open? (not @!open?)) is (swap! !open? not)#2023-09-1218:47xificurCwhat semantics are you trying to capture in
(if (string? trigger)
(dom/text trigger)
trigger))#2023-09-1218:48xificurCin
(if (string? content)
(dom/p (dom/text content))
(content)))))
content will be a string or a clojure function?#2023-09-1218:49grounded_sageYea. Maybe I am going about this wrong. I was just thinking for something simple it would be a string and optionally take a electric dom function when people want to create something more custom#2023-09-1218:50grounded_sageDidn’t know could do swap not. Been writing code like this for ages 🙈 #2023-09-1218:52xificurCI see. Electric functions need a new or dot syntax - (new content) or (content.). We typically uppercase them to remind ourselves#2023-09-1218:53xificurCour error handling and reporting is something to work on. In the meantime when something doesn't work the simplest approach is comment bisecting#2023-09-1218:56grounded_sageI was doing printlns but getting nil. Also the button completely vanishes which is wrapping the trigger parameter. Just pulling up laptop to quickly try what you have shared#2023-09-1218:57telekidnot related to the conversation at hand, but note that (swap! !open? not) is atomic while (reset! !open? (not @!open?)) is not. In general, if notice yourself referencing your atom directly in the second argument of either swap! or reset, you're opening yourself up for a race condition.#2023-09-1219:01grounded_sageYea so when I pass in (dom/div …) in the map assoc’d to the :trigger the button inside of Collapsible disappears. Which I would expect to not be affected.#2023-09-1219:02grounded_sageEven when I wrap the if statement in another div#2023-09-1219:03xificurCcan you share the call? (Collapsible. {:trigger (dom/div ..)}) is not what you'd expect#2023-09-1221:46grounded_sagehttps://github.com/groundedsage/circuit-ui#2023-09-1221:48grounded_sageI'm not sure if this is meaningful to work on and if there is prior work in the Clojure space replicating the work in Radix.#2023-09-1307:08xificurChttps://github.com/groundedsage/circuit-ui/blob/60c6f46c2f438c335495e1a5e7508b902e8d337f/src/app/todo_list.cljc#L178 you can check the thread just after this one why this doesn't do what you expect#2023-09-1308:27grounded_sageYea that let binding is me experimenting trying to figure out what is going on. #2023-09-1310:40grounded_sagenvm. I got news that a UI toolkit is being worked on. I'll stick to my Datahike integration and LLM work#2023-09-1218:49Tommy JollyboatJust coming back to Electric after a long break, so it's possible I've made a stupid error here, but -- Why does this not work as I expect?
(e/defn Div [x]
(dom/div x))
;; later, in an (e/defn ,,, (e/client ,,,))
(Div. (dom/text "hello"))
I expect <div>hello</div>, of course, but it's outputting the rather puzzling
hello
<div></div>
#2023-09-1218:54xificurCdom/text performs an effect of mounting a text node under the current dom node, which is not the div you think it is.
Compare to clojure code (foo (println :bar)), the println runs before foo#2023-09-1218:56xificurC(dom/div (dom/text "hello")) works because dom/div is a macro that expands to code that first mounts a new div, sets it as the current one, then runs the body#2023-09-1218:56Tommy JollyboatAh I see, I assumed it would return a DAG. So dom/text acts on dom/node?#2023-09-1218:56xificurCcorrect#2023-09-1218:57Tommy JollyboatOh, and I see dom/element does the same, so essentially I can't pass around pieces of dom like this. OK!#2023-09-1218:57xificurCto write in this style you need macros. Another approach is to thunk the effect
(e/defn Div [X]
(dom/div (new X)))
(Div. (e/fn [] (dom/text "hello")))
which in clojure would correspond to (foo #(println :bar))#2023-09-1220:33Dustin Getz> I assumed it would return a DAG
e/fn in fact constructs a "DAG as a value" to be passed around and then (new F) splices it in#2023-09-1220:13JHi guys! Does the do run his body in parallel?#2023-09-1220:19Dustin Getzyes, there are a few discussions if you search#2023-09-1220:19Dustin Getz(do (dom/div x) (dom/div)) will render left to right, but if x updates it will update the middle div without touching the final div#2023-09-1220:20Dustin Getzwhich implies a more advanced scenario:#2023-09-1220:21Dustin Getz(do (dom/div (Slow-query. x)) (dom/div (Slow-query. y))) –– the queries run in parallel based on causal dependency on x and y#2023-09-1220:24JThanks @U09K620SG#2023-09-1220:25Dustin Getznote that it's backwards compatible with clojure, except exception semantics#2023-09-1220:26Dustin Getzif the two queries each throw but different exceptions, you'll see surprising things that in edge cases are not backwards compatible#2023-09-1220:30Jwhat do you mean by backwards compatible?#2023-09-1220:31Dustin Getzclojure compatible means if you take a pre-existing clojure library like core.match, and use it inside e/defn, it is guaranteed to work#2023-09-1220:32Dustin Getzwe cannot quite provide that guarantee in the case of exceptions, but in practice, core.match does indeed work#2023-09-1220:33JOk got it.#2023-09-1319:57joshchois there a way to run a sequence of expressions sequentially?#2023-09-1319:58Dustin Getzwhat are you trying to accomplish?#2023-09-1323:14joshchoi have come across situations where i need to call client A, server code B, then client code C. or in general enforce some order to my expressions.#2023-09-1323:17Dustin Getzwe would like to see it in context to see if we can find a better way#2023-09-1323:18Dustin Getzit is possible to sequence today but we don’t commit to keeping that capability#2023-09-1323:34joshchocan't think of any atm, but i'll ping back once that situation occurs#2023-09-1323:34joshchoty!#2023-09-1220:58grounded_sageGratitude to the core team working on electric for their work and answering all our questions.#2023-09-1313:02Dustin GetzWe want to see this ported to Electric if anyone is looking for a cool project https://github.com/opqdonut/malli-edn-editor/tree/main#2023-09-1313:11mmerWould you consider the server holding the schema?#2023-09-1316:07Dustin Getzsure, ideally it shouldn't matter right#2023-09-1316:08mmerI only ask as does electric make sense if no server exists?#2023-09-1317:14xificurCYou can use it client side only, in theory#2023-09-1319:59joshchothis feels very tractable in electric, my standard for how “difficult” a project is has changed#2023-09-1320:41bhurlowI started experimenting with a “CLI” approach for developing and running electric apps: https://github.com/bhurlow/electric-tools. Main motivation here is just preference to not have to copy an existing repo with stuff I don’t need. This grabs the server and build config from the starter app and packages it as a library. Unfortunately due to some cljs specifics, it’s not as plug and play as I’d hoped#2023-09-1320:42Dustin Getzstarter apps are all getting paved fwiw#2023-09-1320:42bhurlownice, how so?#2023-09-1320:43Dustin Getzwe streamlined a lot of the little annoying complexities with the build/ops and some other stuff, i forget#2023-09-1320:43Dustin Getzit unfortunately got paused before completion as i am oversubscribed right now but willget back to it after i do my taxes#2023-09-1320:47bhurlowgot it. happy to review or pitch in on the starter experience, feels like there’s a great opportunity for the long tail of programmers that might not fully grok deps.edn for example#2023-09-1322:49nivekuilhttps://github.com/hyperfiddle/electric/blob/f130ffde95c812bc29edf23e17c18cb74dc94b5d/src/hyperfiddle/electric.cljc#L207 should be listen> not e/listen>#2023-09-1322:54Dustin Getzwill fix, is it breaking something?#2023-09-1322:55nivekuilwell you can't use it because the e alias doesn't exist#2023-09-1322:56Dustin Getzit works if the caller ns binds that alias which of course it does, unless you have bound it to something else#2023-09-1322:57nivekuilI don't understand, it's the only thing in that file that uses e/ , and it was broken for me in my ui file with the usual e alias until I edited that electric source file#2023-09-1322:59Dustin Getzok i will log a ticket, if you want the contributor badge send commit (and we will get to it faster too)#2023-09-1322:56nivekuilmight be interesting, came across this issue in svelte about appendchild vs a faster clonenode method of rendering https://github.com/sveltejs/svelte/issues/3898#2023-09-1323:15nivekuilwhat's the idiom for (.appendChild dom/node my-node) but with proper cleanup?#2023-09-1323:58Dustin Getzm/observe is the default way i think, in the case of electric-dom there is some nuance - mentioning this because i’m not sure what you’re trying to do#2023-09-1406:50xificurCcan you instead (dom/element :my-node-type ...)?#2023-09-1521:41nivekuili want to render to pixi instead of dom, so trying to learn the details a little better. The event handling bits look hard, I don't understand yet why dom event stuff can't be in the dom namespace#2023-09-1523:36Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric.cljc#L183#2023-09-1523:44Dustin Getztell us more about what pixi is and what you are trying to do?#2023-09-1600:49nivekuilpixi is a rendering library, mostly for games, like sdl or flash. it should be straightforward to port electric-dom's rendering model to any other retained mode system. I think this should trivialize making things like .io games#2023-09-1600:52nivekuili saw that comment, but I don't know what it implies in general. I'm looking for a generic pattern of "m/observe this listener, and convert fn callback to e/fn"#2023-09-1600:57Dustin Getzelectric ships an efficient clock on every platform and in the browser implementing that clock correctly depends on the visibility state which is a dom event#2023-09-1600:58Dustin Getzyou can do the naive thing for events with m/observe, it’s the e/fn callback pattern that is problematic, we believe it’s an anti pattern now and it will be deprecated with electric UI5 which is callback free FRP based design#2023-09-1600:59nivekuilthis pattern is in electric-dom, not electric-ui right? does ui5 replace dom#2023-09-1601:00nivekuilis the new design generalizable outside of dom events?#2023-09-1601:01Dustin Getzyes the new design should generalize, however it is currently blocked on differential electric#2023-09-1601:02Dustin Getzwe can give you some old patterns we used to get by if you want, or perhaps you can copy/paste any electric- dom implementations #2023-09-1601:03Dustin GetzUI4 and dom2 are we believe correct and bug free if you understand how to call them correctly , but definitely not idiomatic long run#2023-09-1601:04nivekuili'll wait on the proper way to do it, I've been looking at electric-dom just to try to understand what's going on and I don't get it 🙂#2023-09-1601:22Dustin Getz“hook” is a mechanism to track order in the DAG so that in an e/for-by if elements move around in the collection the underlying dom elements are gracefully swapped and not rebuilt. this is a central functionality in the differential electric rewrite, all this hook stuff is getting paved #2023-09-1410:12PanelI’m using a map rendering lib, openlayers, I set up the map to attach to an id of a a electric dom/div. Render the map and it works.
But everytime the chrome tab goes in the background and the ws close, then the map disappears from the dom.
The dom div is still here but it’s empty.#2023-09-1410:29Dustin Getzwe’re working on what should happen when the tab is not visible (browsers start throttling stuff particularly chrome). fwiw even today it should be reconnecting when it becomes visible again. is it not?#2023-09-1410:30PanelYes it reconnects but the map is not being rendered anymore.
I was think it could be something obvious but I’ll share some code asap#2023-09-1410:32Dustin Getzseems like an integration issue#2023-09-1411:17Panel(ns app.todo-list
(:require
[contrib.str :refer [pprint-str]]
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?(:cljs ["ol/source/XYZ$default" :as XYZ])
#?(:cljs ["ol/layer/WebGLPoints$default" :as WebGLPointsLayer])
#?@(:cljs [["ol" :refer (Map View)]
["ol/layer/Tile$default" :as TileLayer]
["ol/source/Vector$default" :as Vector]
["ol/proj" :refer (fromLonLat)]
["ol/geom" :refer (Point)]
["ol/Feature$default" :as Feature]
["ol/style" :refer (Style Circle Fill)]
["ol/format/GeoJSON$default" :as GeoJSON]])))
(def coordinates-store (atom [[144.99340755998103 -37.879683598879465]
[144.46371303120644 -37.37405653174035]]))
(comment
(swap! coordinates-store conj [144.1988017435401 -37.07493390076035])
(swap! coordinates-store conj [144.1998017435401 -37.07993390076035])
(swap! coordinates-store conj [144.1108017435401 -37.06993390076035])
)
(e/def coord-server (e/server (e/watch coordinates-store)))
(e/defn Vector-update [vector]
(e/client
(doseq [c coord-server]
(. vector addFeature
(new Feature
(js-obj
"geometry"
(new Point (fromLonLat. (clj->js c)))))))))
#?(:cljs (def coordinates [[144.89340755998103 -37.779683598879465]
[144.76371303120644 -37.67405653174035]]))
#?(:cljs
(def features
(clj->js (mapv (fn [coord] (new Feature (clj->js {:geometry (new Point (fromLonLat. (clj->js coord)))}))) coordinates))))
#?(:cljs
(def vector-source
(new Vector #js {:features features})))
#?(:cljs (def webgl-layer (new WebGLPointsLayer (clj->js {:source vector-source
:style {:circle-radius 5,
:circle-fill-color "#ff3f3f"}}))))
#?(:cljs (defn init-map []
(let [xyz (new XYZ #js {:url ""})
tile-layer (new TileLayer #js {:source xyz})
view (new View (clj->js {:center (fromLonLat. #js [#2023-09-1411:36xificurCI think Vector-update can be a cljs function, taking coords as argument?#2023-09-1411:37xificurCwhat is the purpose of the delay? Can't you just call (init-map) directly? I'd try that change first to see if it resolves the issues#2023-09-1411:38nakkaya> Yes it reconnects but the map is not being rendered anymore.
I notice the same behavior on Safari/iOS and long running tabs on desktop. On iOS putting safati to background removes all UI, on desktop it happens after a while.#2023-09-1411:44PanelThe delay was to avoid creating a new instance when the e/defn re-run
I actually moved the map and it's div to an electric component and it works now.
(e/defn Map-instance []
(e/client
(dom/div (dom/props {:id "map"
:style {:height "500px"
:width "500px"}}))
(init-map)))
(e/defn Todo-list []
(e/client
(Map-instance.)
(Vector-update. vector-source)))))#2023-09-1413:34Panel@U09FL65DK If it's just a cljs function, how can it be re-run when more data is available on the server ?#2023-09-1414:36xificurCI think the delay cached the side effect of mounting the map#2023-09-1414:37xificurC(vector-update vector-source coord-server) will re-run when any of its arguments change#2023-09-1414:39xificurCbut looking at the implementation there's a doseq inside, a better approach is to e/for-by on the server so the diffing happens there
(e/server
(e/for-by identity [c coord-server]
(e/client
(. vector addFeature ...))))#2023-09-1501:12PanelI don't understand why the Map-instance electric component will re-run on re-connect.
#?(:cljs (defn init-map [id]
(js/console.log "init-map")
(let [xyz (new XYZ #js {:url ""})
tile-layer (new TileLayer #js {:source xyz})
view (new View (clj->js {:center (fromLonLat. #js [144.9413017 -37.8177539]) :zoom 5}))]
(new Map (js-obj "target" id, "layers" (array tile-layer webgl-layer) "view" view)))))
(e/defn Map-instance []
(e/client
(dom/div (dom/props {:id "map"
:style {:height "500px"
:width "500px"}}))
(init-map "map")))
Console output:
Server timed out, considering this client inactive.
Next attempt in 0.1 seconds.
Connecting...
Connected.
init-map
This is happening after the tab was left in the background (same when simulating offline from devtools)#2023-09-1501:18Dustin Getzmissing cleanup code - those effects need to be managed by m/observe#2023-09-1501:19Dustin Getzelectric behavior today is that on disconnect the app is gracefully shutdown and then on reconnect it will boot again#2023-09-1501:20PanelAlright so I need to find a way to cleanup the map instance when the dom/div is unmounted#2023-09-1501:21PanelWhat should I observe ?
Edit: reading -> https://github.com/hyperfiddle/electric/blob/5a3799bdca4f70aed0dc51ac98ff9728dbffacae/src-docs/user/electric/electric_lifecycle.clj#L18
Thanks for the response Dustin, I read so many responses from you on this channel that says "use m/observe" I should have started there.#2023-09-1501:34Dustin Getzyeah if you search m/observe you will find lots of discussions#2023-09-1416:54Jhttps://x.com/redplanetlabs/status/1702363303908089870?s=46&t=FLgxpB8u-zgNc4wlhiD7Gw#2023-09-1419:27telekidhttps://www.youtube.com/watch?v=iVFmuoGvEZM. It was pretty easy to magically ensure that the OIDC id_token and application_tokens are always up to date from the perspective of the application, to propagate user info updates (like name changes) all the way to the UI, sign the user out on token expiration, etc.#2023-09-1500:16PanelCan e/def be inspected at the repl ?#2023-09-1501:24Dustin Getznot really, you can deep def or you can use the test electric entrypoint in an RCF test#2023-09-1501:24Dustin Getzthe problem is that electric anything can only be resolved from an electric program (not valid in clojure as clojure is not aware of electric) so to use the REPL you have to bring your own electric entrypoint (see our unit tests)#2023-09-1500:47joshchoI sometimes get Unable to resolve symbol errors that are really hard to pin down. Why does this occur?#2023-09-1500:50joshchoI suspect it's because of incorrect require, or something along those lines? I have faced it being really confused why the symbol isn't being resolved, only to realized that there is some error regarding compilation somewhere else.#2023-09-1501:20Dustin Getzpost an example stacktrace please #2023-09-1501:22Dustin Getza common gotcha (that i’ve been meaning to improve) is when a stale browser tab reconnects to a fresh server, especially a stale tab from an entirely different electric app, which will (because the DAG is initialized by the client) start resolving stuff on the server classpath that isn’t there#2023-09-1501:23joshchoThis is totally not helpful, but I posted this after I resolved the problem a couple hours ago. I don't think I can reproduce it. Sorry about that. I will return to this if I face something like it again.#2023-09-1501:27Dustin Getzi also don’t yet trust shadow-cljs setups that are not explicitly booted by the app/repl via the clojure shadow API, see the documented shadow setup in the starter app user.clj#2023-09-1501:29joshchoHm, so the one via cider is not trustworthy?#2023-09-1501:32Dustin Getzin theory the editor plugins can work but i’ve seem unstable compilation happen with them in ways i don’t understand, what i know for certain is that when i explicitly control compilation i have zero issues#2023-09-1501:33Dustin Getzim working on understanding calva better , hope to have a validated calva setup soon that is more tightly integrated than “run shadow from user.clj” but i haven’t had time yet to go deep#2023-09-1501:43joshchoyeah I have seen things happen where I'd restart emacs and compilation errors would go away.#2023-09-2403:43nivekuilcontext: trying to avoid designing a macro-based api with a dynamic var to hold context, but I think it might be the only way to get good ergonomics, since we're stuck with compile-time dependency injection#2023-09-2411:05Dustin Getzwhat is compile time DI, are you referring to the Electric compiler?#2023-09-2411:06Dustin GetzThat opinion that Dynamic scope is RT is controversial, i’ve mostly had it beaten out of me (by Leo and others). Regardless of if it is RT i think it’s still great, it just interacts terribly in ordinary clojure with lazy-seq#2023-09-2417:34nivekuilso the solution to making dynamic scope not suck is to not do it in "ordinary clojure", so the electric dyn vars are only bound within the electric program?#2023-09-2417:36nivekuilarguably lazy-seq is the bigger mistake.. it's a really idiosyncratic thing to be idiomatic, other languages get by fine without them#2023-09-2418:08nivekuilI guess this is why you cannot bind dynamic vars inside electric programs. already ran into that trying to use with-out-str#2023-09-2422:17Dustin Getzan important difference between Electric dynamics and Clojure dynamics is that Electric dynamics have the Electric RAII lifecycle, making them more or less equivalent to Component/Mount as i believe we discussed last month#2023-09-2422:19Dustin Getzand since Electric code does not make use of the clojure seq abstraction at all (due to it not being reactive), the bad lazy seq interaction is avoided#2023-09-2422:21Dustin GetzClojure dynamics have other problems with respect to how they interact with JVM concurrency and specifically about how Clojure’s “binding conveyance” between threads has serious shortcomings, Leo wrote a rant about this that I’ll have to go find. Again, Electric doesn’t use Clojure’s concurrency so those interactions are also avoided#2023-09-2422:25Dustin Getzwith respect to binding Clojure dynamics from Electric programs (breaking with-out-str) I need to go look at our internal issue tracker to remember our stance on that and the details/options#2023-09-2506:57xificurCre electric -> clj(s), reactive vars are conveyed where possible
(e/def db (e/watch !conn))
(e/defn App []
(d/q [] db) ; obviously works, passed as a positional argument
(#(prn db)) ; also works, we analyze and convey
(defn my-transact [] ; not electric
(d/q [] db)) ; not possible to convey
#2023-09-2507:30leonoelmy opinion on lazy sequences and dynamic vars in clojure :
• the dynamic var idea started with good intentions but binding conveyance completely ruined it. Without binding conveyance, dynamic vars are equivalent to thread local state, which has some valid use cases. Using thread local state for DI is a bad idea because it breaks everything that relies on referential transparency - not only lazy sequences, really all kinds of deferred evaluation. If thread local state is what you need, use ThreadLocal because dynamic var performance is terrible (due to binding conveyance, even if you don't use it). To understand why binding conveyance is bad, consider this snippet : (with-out-str (future (println "hello")))
• lazy sequences are not bad but it should not be the default because laziness+memoization is almost never what you need, you generally need one or the other but not both, so you pay the performance overhead for nothing#2023-09-2507:47nivekuilthat's true, and in the two places I actually use lazy seqs the chunking works against me too#2023-09-2507:48nivekuilwe need to spread the word about eduction#2023-09-2507:48leonoeleduction rocks#2023-09-2423:04nivekuilhow is compile performance for people? I feel like i've done something wrong, getting ~9s compiles for a small test project#2023-09-2423:43nivekuilnarrowing it down a bit, this seems weird#2023-09-2423:46nivekuilseems like every dom/on adds 2 seconds? that doesn't seem right#2023-09-2500:32Dustin Getzi’ve never seen that#2023-09-2500:35nivekuilhow can I debug the compilation more?#2023-09-2500:36Dustin Getzyou have proven that it is specific to this project?#2023-09-2500:40nivekuilah, it's actually the varargs that's triggering it#2023-09-2500:40nivekuilvarargs + dom/on#2023-09-2500:40nivekuil(e/defn Foo [body & rest]
(ed/on "click" (e/fn [_])))
4.33 seconds#2023-09-2500:40nivekuil(e/defn Foo [body]
(ed/on "click" (e/fn [_])))
1.63#2023-09-2506:51xificurCtoday varargs expand https://github.com/hyperfiddle/electric/blob/0842e64e895b260707a532068dc7f35c147fb65f/src/hyperfiddle/electric.cljc#L247arguments. Reducing the number reduces the compile times, but we don't have a dynamic fallback case today so I have to make the cutoff somewhere#2023-09-2507:43nivekuila single instance of e/defn .. & with dom/oncurrently doubles my build time. IMO it is completely unusable right now#2023-09-2508:59xificurCthank you for the feedback, we'll get back to this at some point#2023-09-2510:21Dustin Getzwhat Peter is saying is that we probably want to land incremental compilation first which is our platform for future work on the build time#2023-09-2510:22Dustin Getzthe specific interaction with dom/on is also surprising, needs more analysis on our part #2023-09-2510:23Dustin GetzSurely we can tune the varargs constant for now, and it feels like something O(n^2) may be happening as well#2023-09-2511:00Dustin GetzImmediate workaround: lift to arity-1 function that takes a single list and destructure the list#2023-09-2507:11mattiasIs there a way to use Electric inside chrome extensions (e.g. on the panel or sidebar that execute in chrome devtools). I have a setup that works with shadow-cljs and rum, and I'd like to swap it out with Electric.
Electric's shadow-cljs.edn has :target :browser but I also need to use :target :chrome-extension . I have a :browser build for creating the Electric backend (and starting up Chromium via Playwright's Java API which also loads the chrome extension from the shadow build). The :browser build works fine on localhost, and the bare bones :chrome-extension build works as well.
What I need now is a way to compile a devtools_sidebar.js file in the :chrome-extension build that would connect to the web/Electric backend. In addition to devtools_sidebar.js knowing how to properly connect to the backend, the cljs-runtime files that are created for :chrome-extension build would need to have the necessary Electric dependencies. I guess you could generalise this question to "how to connect to an external Electric backend from a different cljs build?"
Is this doable in principle? If so, could you give me additional pointers where to look next to understand how to make this happen? Or if anyone has time to help out, I could grant access to my existing code example. Any help is much appreciated!#2023-09-2507:36Geoffrey Gaillard> How to connect to an external Electric backend from a different cljs build?
>
Today this is not possible.
It might be possible in the future, but it requires more research.
Do you really need to share the server side process between the page and the extension. I assume you actually want to share state? If so, you can have two electric programs, one for the page and one for the extension, so two builds. If both electric programs runs on the same server jvm, then they can easily share state. If they share state, they can react to each other's state changes.#2023-09-2507:46mattiasTwo separate backend processes / shadow builds is fine as well. Is there a way to make a shadow build that has :target :chrome-extension ?#2023-09-2508:54Geoffrey GaillardWe haven’t tried :target :chrome-extension yet.
Editing your shadow-cljs.edn and replacing :target :browser by :target :chrome-extension should do the trick.
https://github.com/groundedsage/shadow-cljs-chrome-extension-hello-world/tree/main.#2023-09-2509:01grounded_sageI’m also planning to build a chrome extension soon so I’m following this. #2023-09-2509:02grounded_sageGoing to be deploying a native app with webview and chrome extension. So might have to do the double server you said @U2DART3HA #2023-09-2513:05mattiasWith this approach I'm getting Unexpected protocol exception when parsing the URL :port or something like that works? What would be the correct hardcoded string to try if my app is accessible at server-url hasn’t been designed with chrome extensions in mind. Feel free to patch it (or improve the design) and submit a PR#2023-09-2513:38mattiasHardcoding the response of server-url to "" actually makes it work, which is pretty awesome!
But I'm not sure how a good PR for this would look like, because the original URL does not indicate what the host/port/protocol of the backend actually is. Sometimes it would be localhost and http, but it might be some remote url over https as well. Would it make sense to be able to (optionally) pass it as a build argument?#2023-09-2514:36Geoffrey GaillardBuild argument would technically work. Though I’m not sure if it should be hardcoded.
I’d assume a production-ready chrome extension to:
• fetch initial configuration from an API.
• or have the page postMessage the config to the extension
• or always connect to chrome.storage.sync.set and updated to different Electric frontends (devtools sidebar and panel, content script) with chrome.runtime.sendMessage .
3. Might need to have some initializer (background page?) set the default backend in chrome.storage.sync and emit the initial backend URL with chrome.runtime.sendMessage .
4. Inside Electric frontends, have chrome.runtime.onMessage listeners for changes in the backend URL. If it is detected, it reconnects - is that possible? Before it receives a message, it would not connect anywhere.
Not sure if it's feasible with the current architecture though.#2023-09-2609:36Geoffrey Gaillard> If it is detected, it reconnects - is that possible?
Yes it is possible.
On first message, onMessage would call e/boot to start the electric program
On second message, onMessage would stop the running electric program and restart it by calling e/boot again with the new domain#2023-09-2707:04mattiasI actually got the following setup working, which is the initial thing I asked for:
1. There is a clj backend process (namespace proj.backend ) that starts an Electric server and has two shadow-cljs watches (one for the :web build frontend and one for :ext build js). In addition to starting Electric and shadow, it starts a nrepl server, where I connect with Calva. The process can start a Chromium browser with the chrome extension from the :ext build.
2. The :web build has :modules {:main {:entries [proj.web.frontend] :init-fn proj.web.frontend/start!}} , so I can access the web frontend in a browser.
3. The chrome extension manifest.edn defines outputs like :devtools-panel {:init-fn proj.ext.devtools-panel/start! for the DevTools panel and sidebar (and could probably work for content script as well).
4. Because I patched server-url to always returns ws:0.0.0.0:8888 which is the URL that proj.backend binds to. So now the web frontend, devtools sidebar and devtools panel javascripts connect to the same backend. Which for me makes sense, because they work as a unified whole (you would often use the extension to interact with the web frontend, so syncing state makes sense). proj.backend has an in-memory XTDB database, and changes from this get propagated to all the different frontends. Not sure if it would behave the same with regular atoms in the backend, or it would create separate state per ws connection (so one for sidebar, one for panel, and one for the web frontend).
I guess the reason the single backend approach here works is because I've hardcoded the server-url response? I also tried an approach where I have separate backend processes for the :web and :ext builds with their respective shadow-cljs watches, but (1) shadow-cljs did not allow 2 simultaneously running shadow processes, and (2) I would need to use an XTDB that is not in-memory (is in a separate process and I'm listening on the transaction log in each (backend ?) process). I would need to do (2) soon anyway when scaling things out, though.
Will keep exploring this to see if I reach some rough edges, but so far it's pretty cool to get auto-reload and auto-saync between my web app and chrome extension.#2023-09-2508:45nivekuilI pass around a e/fn , and when I call new on it crashes the reactor
2023-09-25T08:43:06.039Z machina ERROR [hyperfiddle.electric-httpkit-adapter:326] - Websocket handler failure. nil
java.lang.Thread.run Thread.java: 1623
java.util.concurrent.ThreadPoolExecutor$Worker.run ThreadPoolExecutor.java: 642
java.util.concurrent.ThreadPoolExecutor.runWorker ThreadPoolExecutor.java: 1144
java.util.concurrent.FutureTask.run FutureTask.java: 317
java.util.concurrent.Executors$RunnableAdapter.call Executors.java: 577
org.httpkit.server.LinkingRunnable.run RingHandler.java: 156
org.httpkit.server.WSHandler.run RingHandler.java: 189
org.httpkit.server.AsyncChannel.messageReceived AsyncChannel.java: 153
clojure.core/partial/fn core.clj: 2641
hyperfiddle.electric-httpkit-adapter/handle-electric-ws/on-receive electric_httpkit_adapter.clj: 75
clojure.core/comp/fn core.clj: 2586
missionary.impl.Observe$1.invoke
missionary.impl.Sample$3.invoke Sample.java: 146
missionary.impl.Relieve$1.invoke Relieve.java: 86
missionary.impl.Reactor$1.invoke Reactor.java: 480
missionary.impl.Reactor.event Reactor.java: 398
missionary.impl.Reactor.propagate Reactor.java: 242
missionary.impl.Reactor.touch Reactor.java: 222
missionary.impl.Reactor.pull Reactor.java: 183
missionary.impl.Sample$Process.deref Sample.java: 34
missionary.impl.Sample.transfer Sample.java: 84
missionary.impl.Util.apply Util.java: 29
clojure.core/partial/fn core.clj: 2648
clojure.core/reduce core.clj: 6899
...
hyperfiddle.electric.impl.runtime/parse-event runtime.cljc: 771
clojure.core/reduce core.clj: 6899
...
hyperfiddle.electric.impl.runtime/eval-tree-inst runtime.cljc: 736
...
java.lang.NullPointerException: Cannot load from object array because "xs" is null#2023-09-2508:46nivekuilthis simple example does NOT repro but it's the general idea, will keep digging
(let [!x (atom nil)
x (e/watch !x)]
(when x (new x))
(eui/button (e/fn [] (reset! !x (e/fn [] (ed/text "i die"))))
(ed/text "kill me")))#2023-09-2508:51nivekuilnote that the thunked effect (rendering the text) does go off before the reactor crashes#2023-09-2510:28Dustin Getzis a lambda nil? do you have unserializable reference transfer warning? lambdas do not transfer today which is quite easy to accidentally cause using current semantics, you need to be aware of it#2023-09-2510:29Dustin Getzonce you confirm e/server is not in play,#2023-09-2510:30Dustin Getzwe have a nasty deep bug called “when true” bug which can cause the body of a when to see an impossible value, it is fixed in next electric but not in master, i’ll get more details for you later today from our issue tracker#2023-09-2510:32Dustin Getzthere are workarounds#2023-09-2511:18nivekuilno e/server or serialiization errors. it's not a priority for me, actually thinking about a workaround led to a better data model: dynamic state systems don't need to handle singletons, since those are always known up front and can just be def 'd#2023-09-2511:27Dustin Getzi checked our tracker, the "when true" bug is an interaction with Pending, so if you have no e/server then I don't see how this is connected. electric-ui4/button perhaps could emit a pending (i just checked and didn't see one but the infrastructure ui4 uses is really complicated). We can rule it out by writing a barebones dom button with only electric-dom2#2023-09-2511:27Dustin GetzI don't understand your comment about singletones#2023-09-2511:30nivekuilelectric has reactive static state. For dynamic state you use something like datascript or an atom holding a collection, but those aren't reactive. I've been struggling on figuring out a good api for my reactive dynamic state system to deal with cardinality and it turns out I can just leave all the cardinality-1 stuff to electric#2023-09-2511:32nivekuilanyway, my planned workaround was to put the e/fn in an atom like my simple example but it turns out that still crashes it.. will have to dig further tomorrow#2023-09-2511:52Dustin Getzwe are disaligned on terminology#2023-09-2511:54nivekuilyou can (def todos-count (atom nil)) and set that at runtime and that's as granular as you'll ever want. you can't (def all-todos) and have granular reactivity per todo#2023-09-2511:56Dustin Getzdifferential electric maybe can do that? If you are ingesting batch data then someone has to diff, in Electric 2 (today) e/for does the diffing in Electric 3 diffing is built into e/watch (details still firming up) and then diffs flow through the DAG not collections#2023-09-2512:00nivekuili am too tired to think right now, but can you efficiently say "do this when :todo/id 1's :todo/title changes"? I was thinking differential electric is like being able to serialize flows#2023-09-2519:09Dustin Getzi think i need a clearer example but likely yes, differential electric is insane (good insane)#2023-09-2519:13Dustin Getzfwiw, putting e/fn in an atom is valid and reasonable and can be idiomatic, but it's also twisted enough that I am unsurprised if you find Electric evaluation bugs here#2023-09-2519:14Dustin Getzi'm not sure if we have any actual instances of that in any of our repos currently, maybe a couple unit tests#2023-09-2519:22nivekuilhere's my extremely concise DSL#2023-09-2519:22nivekuilthis does two pathom queries on mount and reactively subscribes to a datalog store. first [::global ::session/current-user session/current-user]#2023-09-2519:23nivekuilwhere session/current-user is {:user/id 1} , then the second one does [[_ :user/id 1] [_ :user/name user/name] ...#2023-09-2519:24nivekuilright now my reactive datalog comes from odoyle, a runtime reactive network. you're telling me that electric will be able to do this at compile time?#2023-09-2519:25nivekuilit was super cool when I realized that you don't need EQL or datalog joins at all, you can just do them in electric and you get fine grained reactive joins for free#2023-09-2519:26nivekuilnow you're telling me that https://github.com/leonoel/missionary/commit/187ff9eb630b4281d4b1eb1d33f165a0aa5535cf implements differential dataflow in <200loc? what#2023-09-2519:27Dustin Getzhttps://github.com/hyperfiddle/electric/compare/5a3799bdca4f70aed0dc51ac98ff9728dbffacae...46f9a0dc39f10afcb1fb80d6ff58c50f2ea04055#2023-09-2519:27Dustin Getzgood luck#2023-09-2519:31Dustin Getzit's a missionary plugin, it doesn't depend on missionary because missionary flows are "dependency free" you can construct missionary compatible flows out of just lambda#2023-09-2519:34nivekuilnested incremental sequences?#2023-09-2519:36Dustin Getzyes#2023-09-2520:04nivekuilwell if you can insert triples into a big array and efficiently get back the diffs, I guess you just need an {a {e #{flow}} index and you can transfer those diffs to the UI efficiently#2023-09-2520:06nivekuilhttps://github.com/hyperfiddle/electric/blob/0842e64e895b260707a532068dc7f35c147fb65f/src/hyperfiddle/incseq.cljc#L1203 was the only example I looked at btw, no luck needed#2023-09-2520:07nivekuilin fact if you have that index you can just hydrate from it too.. fuck#2023-09-2511:56nivekuillooking at adding transit types, https://github.com/hyperfiddle/electric-examples-app/blob/main/src/wip/demo_custom_types.cljc says the client only needs read handlers?#2023-09-2512:00Geoffrey GaillardNo, sorry for the confusion.
Both client and server can implement both read and write handlers.
If your use case requires the client to send custom types to the server, then you’ll need to implement client write and server read.#2023-09-2512:01nivekuilthanks, for a moment I thought electric was really magical#2023-09-2520:46Jordan CalderwoodIs it possible to set css vars from electric dom?
(dom/div
(dom/props
{:class "slider-container"
:style {"--num-sizes" num-sizes}}) <----- this bit here
...#2023-09-2520:58Geoffrey Gaillarddom/props doesn’t support setting CSS vars yet.
In the meantime, this will work:
(dom/div
(dom/props {:class "slider-container"})
(.setProperty (.-style dom/node) "--num-sizes" num-sizes)
(e/on-unmount #(.removeProperty (.-style dom/node) "--num-sizes")))
#2023-09-2520:59Jordan Calderwoodamazing. Thanks for the workaround!!#2023-09-2601:04joshchoCurious what people think of Svelte from the perspective of Electric#2023-09-2601:06joshchofrom a cursory look, runes (new in svelte 5) feel like incidental complexity/artifacts#2023-09-2605:41HendrikGiven how electric manages updates. Are there already best practices how to manage purely frontend state with electric? I can think of 3 alternatives.
1. one global atom. Having an e/watch in any clientside component. pro: simple to implement, con: triggers some reevaluating in every component on state change
2. split state into many atoms. Pro: absolute minimum recalcultion on state changes. Con: clutters state
3. fulcro like global atom. state is injected as fn arg into component. Pro: good reasoning about state. con: “props drilling react like problems”
Personally, I would go with method 1. How would you do it with electric?#2023-09-2606:08HendrikI just realized that I can do another 4th alternative.
(def !state (atom {}))
(e/def state (e/watch !state))
(e/def topic1 (:topic1 state))
(e/def topic2 (:topic1 state))
(e/defn MyComponent []
(dom/div (dom/text topic2)))
This has the benefit of having state in 1 place. No props drilling is needed in the component. Nested components are decoupled. State could be even a normalized db like in fulcro with denormalization for components. This 4th alternative seems the right way to go#2023-09-2607:09xificurCall of the above work. I'd probably store it in datascript#2023-09-2607:15nivekuilI wrote my own, first with odoyle+asami for reactive inline datalog, now in an EQL style heavily inspired by fulcro. it works well though I need to rewrite it yet again on top of differential electric soon. Note that your option 4. is basically how react renders but even less efficient since you don't get to skip subtree diffs#2023-09-2607:17nivekuilalso, don't underestimate the importance of transactions. i've ran into some very confusing stacktraces in electric already because of badly timed state updates#2023-09-2607:23xificurC> since you don't get to skip subtree diffs
What do you mean by skipping subtree diffs? In option 4 (dom/text topic2) will re-run only if topic2 changes. If the !state atom updates but (:topic2 state) returns the same value (same via clojure.core/=) downstream dependencies will skip all work#2023-09-2607:27nivekuilit needs to run every diff (checking if (:topicN state) returns the same value) every time !state updates since he doesn't want to do prop drilling#2023-09-2607:30nivekuilwith react you diff mounted components, and if the parent doesn't render no work is done for its children#2023-09-2607:40nivekuilbasically you want to diff as far upstream as possible. that's why react's low level built-in diffing (vdom<->dom) isn't that impressive#2023-09-2608:02xificurCchecking if (:topic2 state) is = when there have been no changes on that branch succeeds at pointer comparison, I wouldn't worry about that cost#2023-09-2608:04xificurCin our ideal world view diffing is accidental complexity introduced at the edges of your system where you have to interoperate with third parties that send you collections. e/for diffs and in next electric version everything will be differential#2023-09-2608:05nivekuilit is very cheap and react (in the happy path) is plenty fast, though not quite free, so you do get something of a "death by a thousand cuts" effect. definitely not relevant for most apps#2023-09-2608:06xificurCe.g. if rama can send fine grained updates (CUD) we should be able to integrate that seamlessly and achieve "perfect" performance#2023-09-2608:10nivekuildiffing as a threshold problem makes sense, it's like a breakdown of trust/communication so you have to verify first which has a cost#2023-09-2610:27Dustin Getzto address the q of state management: I hesitate to prescribe solutions here yet, we are still learning - electric is different, and differential electric (coming this year) is even more different and extremely powerful#2023-09-2610:29Dustin GetzUI5, the optimistic update forms library, has a new flow-based pure functional state management approach (no callbacks). it is blocked on differential electric#2023-09-2610:32Dustin Getzmy instinct is that callback-free flow composition is the path forward for electric state management but in the meantime we are stuck with change callbacks#2023-09-2610:33Dustin Getzalso since the UI5 workstream is blocked, this is all just a hypothesis until we see it working#2023-09-2618:41siddharth yadavIs it possible to do e/server or e/client from a react or reagent component?
I am trying to make a svg playground UI which has codemirror, prosemirror and a few other type of nodes in the playground. I currently have this in working state using re-frame and reagent but I want to move over to electric.#2023-09-2618:41Dustin Getzno, those special forms are only valid inside the electric macro contexts#2023-09-2618:42Dustin Getzthe best you can do is use electric to pre-fetch the data and pass it into the foreign context#2023-09-2618:43Dustin Getzor implement some sort of subscription machinery form the foreign context to send an event back into the electric context to signal a request for more data#2023-09-2618:46siddharth yadavoh ok thanks for the quick reply Dustin.
I really like working with electric, I find it much more intuitive to use than reagent and any other framework. I think I should implement the svg playground in electric itself not sure the complexity of doing so.#2023-09-2618:47Dustin Getzcan i see a screenshot of what you have?#2023-09-2618:54siddharth yadavThis is screenshot I have from the working code in reagent, the blue part is codemirror editor when we run the code we get output in the white area. We can then pass this result to another such node. The playground in which these nodes exists is build on reactflow.
Sorry I could not share a better screenshot because I don't have a running build as of now.#2023-09-2707:46xificurCbridging react is a common question, here are some links
• https://github.com/hyperfiddle/electric/blob/c910935c7068ee5af457a64157a6e12923b8393e/src-docs/user/demo_reagent_interop.cljc
• https://github.com/KevinTung/electric-testing-1/blob/main/src/app/todo_list.cljc (@U03RQQ2L8LE’s repo, I helped)
There are rough edges we can help clean up. If you hit a specific scenario where you get stuck I'll be glad to help#2023-09-2708:18KeinHere are some context:
https://clojurians.slack.com/archives/C7Q9GSHFV/p1693321123284739
https://clojurians.slack.com/archives/C7Q9GSHFV/p1693840471256199
https://clojurians.slack.com/archives/C7Q9GSHFV/p1695109763946729
I ultimately used a normal event atom to store and trigger update. It functioned well .#2023-09-2708:37siddharth yadavThanks @U09FL65DK and @U03RQQ2L8LE! I will try to use this and ask if I hit some blocker.#2023-09-2720:26weibtw if anyone is interested in wrapping React Flow or porting it to Electric, I'd be interested as a consumer and/or contributor. it's a great fit for Electric!#2023-09-2720:42CanHello! How can I get value on datepicker element in electric?
i.e.:
(ui/date
(dom/on "change" (fn [v] (swap! !state assoc :category v)))
(dom/props {:style {:name "date" }}))
Which is the correct usage of getting the date from datepicker and assign that value to an atom.#2023-09-2722:49Vincentis there a ui4 date component? o_O#2023-09-2805:59Can[hyperfiddle.electric-ui4 :as ui]
yes 🙂#2023-09-2900:51nivekuilout of curiosity, has the electric team looked at e-graphs?#2023-09-2900:52Dustin Getzhttps://en.m.wikipedia.org/wiki/E-graph#2023-09-2900:52Dustin Getzno , looks optimization related which we mostly haven’t gotten to yet#2023-09-2900:52nivekuilhttps://egraphs-good.github.io/#2023-09-2907:10nivekuilnot using https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/history.cljc, but looking at it I'm wondering how you would restore scroll position on back/forward (life is very painful without this). You need some top-level way of knowing when everything has been rendered before you can set the scroll position. how do you do that when everything is rendered eagerly? is thunking ~@body always enough to perfectly defer all effects?#2023-09-2910:26Dustin Getzgood q, we will need to think about this#2023-09-2910:29Dustin Getzpossibly electric-dom will need to be changed, there have been some chats internally about doing batched dom updates which is related#2023-09-2910:29Dustin Getzdifferential electric is first#2023-09-2907:52CanHello! Is there any example in Electric we generally use JavaScript for learning.
I.e. I picked a basic page (one input one button) on the internet and tried to write the same code in electric. But while same code working nicely in the HTML compiler My code in electric my javascript validation function doesn't work.
What is wrong here?
https://gist.github.com/Bariscanates404/4549b116fe44b3da1fd82315fedad4c9#2023-09-2907:57nivekuilwhy do you think it doesn't work? have you tried printing the value of username on click?#2023-09-2907:59Geoffrey Gaillard(def usernameInput (.getElementById js/document "username")) executes on JS file load (when the page loads)
At the time it runs, the #username HTML is not mounted yet#2023-09-2908:00CanI giving an invalid number input i.e. "12" it must be a minimum of 3 characters. That doesn't show me an error as in HTML#2023-09-2908:01nivekuilwell, geoffrey gave you the answer 😛 but generally you can narrow down what the problem must be by printing stuff out that you think should be run#2023-09-2908:02Geoffrey GaillardAlso, while Electric compiles down to JS, their evaluation model are different (imperative vs reactive). You cannot directly translate idiomatic JS into idiomatic Electric. Let me get back to you with an idiomatic Electric code snippet#2023-09-2908:03nivekuilelectric creates the elements dynamically when the js is run, while the browser will read the html first and create those elements before the js is run#2023-09-2908:23Can@U2DART3HA Thanks, I want to learn JavaScript usage in electric I think i will need it always 🙂#2023-09-2908:45Geoffrey Gaillard#2023-09-2908:51Geoffrey GaillardNotes:
• + in clojure only does addition. Use str to concatenate strings
• < , >, <=, >= can take more than two args. Useful to check ranges: “x is between 3 and 12 included” : (<= 3 x 12)
If you want to call addEventListener manually, you can do so on dom/node:
(dom/button ; in this block, the current DOM node is bound to `dom/node`
(dom/text "Click me")
(.addEventListener dom/node "click" handle-click))
(defn handle-click [^js event]
(js/alert "Button was clicked"))
#2023-09-2908:51CanThanks! Starting study on it.#2023-09-2908:52CanThat is very valueable help for me. 🙂#2023-09-2910:59Dustin GetzI would consider doing all imperative js-style code in ordinary clojurescript until you have a reason not to. Just use defn instead of e/defn#2023-09-2911:00Dustin GetzI guess you were already doing this, maybe this comment isn't helpful#2023-09-2916:07Dustin PaluchHow would you remove the reactivity from TwoClocks, such that the current time on server/client is displayed once immediately but then not updated? Or perhaps, add a button that turns the streaming updates on or off?#2023-09-3009:25mattiasI need to call a JS function from Electric and receive the return value via a callback I'm passing to the function. I need that value "in Electric" so that I can perform some action with it in the backend. I can achieve this by creating a temporary JS atom and watching that, but perhaps there is a better way to do this with missionary?
(e/client
(let [!res (atom nil)]
(js/chrome.tabs.sendMessage js/chrome.devtools.inspectedWindow.tabId
(clj->js {:action "do-sth"})
#(reset! !res (js->clj %)))
(when-let [new-res (e/watch !res)]
(e/server (println "!res changed" new-res)))))#2023-09-3009:52mattiasActually, I either need to wrap it in another e/server call
(e/server
(e/client
(let [!res (atom nil)]
(js/chrome.tabs.sendMessage js/chrome.devtools.inspectedWindow.tabId
(clj->js {:action "do-sth"})
#(reset! !res (js->clj %)))
(when-let [new-res (e/watch !res)]
(e/server (println "!res changed" new-res))))))
or do some other stuff in the server, like another print statement
(e/client
(e/server (println "sth"))
(let [!res (atom nil)]
(js/chrome.tabs.sendMessage js/chrome.devtools.inspectedWindow.tabId
(clj->js {:action "do-sth"})
#(reset! !res (js->clj %)))
(when-let [new-res (e/watch !res)]
(e/server (println "!res changed" new-res)))))
Otherwise I don't see the backend print statement "!res changed"#2023-09-3010:22Dustin Getzanswering from mobile - this is a common q, try searching the history#2023-09-3021:20joshchoWhat is differential electric?#2023-10-0110:55xificurCThe upcoming next version which will internally work on differential data.#2023-10-0111:08joshchoWdym by differential data?#2023-10-0111:16xificurCDiffs. E.g. instead of sending a collection you send "update 5th element to x"#2023-10-0111:37Dustin Getzdifferential electric is natively streaming, it works on streaming event sources (like LLM completions) natively. It fixes several implementation issues as well that were preventing Electric e/for from achieving optimal network IO and introducing unexpected latency #2023-10-0112:43Dustin Getzdifferential electric is a strong enough foundation to build ultradynamic responsive UIs like say microsoft word. By operating on flows of diffs instead of flows of values it basically means UI code never has to fullscan a collection, even large lists can be rendered without fullscan. Unlike e/for-by today which traverses the full list everytime any element changes#2023-10-0112:45Dustin GetzIt is probably also a strong enough foundation to build reactive databases like Rama and Materialized in a small amount of LOC, though this is not our mission. Such a database could have an electric client/server split right through the middle of the query engine and indexes, potentially solving the local-first problem once and for all. In this worldview basically the DB and UI are unified and interwoven in the same way that client/server are in electric#2023-10-0117:13joshchoso to the end user, extremely responsive and fast ui, and the programming model doesn't need to worry about how big our collection is? are there other benefits for the user?#2023-10-0117:14joshchoah so this will be huge for streaming with llms. i thought some parts of electric were already differential#2023-10-0117:34xificurCe/for is, but the differential version will fix some issues it has#2023-10-0311:34joshchoWill differential also be an option for strings, e.g. streaming building up a string? Workaround is trivial but#2023-10-0311:52xificurCcan you describe the use case?#2023-10-0312:17joshchollm streaming, building up a string via str#2023-10-0312:18joshchobut a workaround is easy with a list of string tokens instead#2023-10-0312:34xificurCif the string is rendered as text only you can just collect them and dom/text them in. If it's parsed and a DOM structure is built differential electric will help with that. No snippets to share on that yet though 🙂#2023-10-0312:40Dustin GetzYes arbitrary differential collections including accumulating collections of strings (i.e, for rendering a table view based on a stream of strings)#2023-10-0312:40Dustin Getz@U09FL65DK I dont understand your statement#2023-10-0312:59xificurCwhich part? Also, the question was "streaming building up a string", not accumulating collections of strings#2023-10-0109:30nivekuilwhy are focused/hovered implemented like that? I thought e/fn was the dual of new , isn't it redundant
(e/def Hovered? "Returns whether this DOM `node` is hovered over."
(e/fn* []
(->> (mx/mix
(e/listen> node "mouseenter" (constantly true))
(e/listen> node "mouseleave" (constantly false)))
(m/reductions {} false)
(m/relieve {})
new)))#2023-10-0110:53xificurCThe new here is joining a missionary flow#2023-10-0111:32Dustin Getzthe expression captures the current dom/node in dynamic scope at the time new is called#2023-10-0111:34Dustin Getzperhaps it is better phrased as a flow constructor taking explicit dom/node as argument, but then the userland syntax changes, perhaps still might be better#2023-10-0115:32Dustin GetzI may be confused regarding the exact semantics of the message I marked, am clarifying, will report back#2023-10-0117:51nivekuilso e/fn never captures bindings?#2023-10-0118:20xificurCbinding has dynamic extent. Lexically scoped values with let are closed over and captured#2023-10-0220:22Dustin GetzMy followup answers had errors, I just deleted them to avoid mass confusion#2023-10-0220:23Dustin GetzThe reduced version (with the e/fn and the new cancelled out) didn't actually work due to capturing dynamic scope at different moments. Perhaps you saw this, if so sorry!#2023-10-0221:39nivekuili did but i don't remember so now i'm as confused as can be. can you give an example of when the behavior is different#2023-10-0222:47Dustin Getzat this point you should just refactor it however you like and try and hopefully it will be clear#2023-10-0308:25xificurCA pure missionary solution would look like
(defn hovered? [dom-node]
(->> (mx/mix
(e/listen> dom-node "mouseenter" (constantly true))
(e/listen> dom-node "mouseleave" (constantly false)))
(m/reductions {} false)
(m/relieve {})))
and one would call it as (new (hovered? dom/node))
If we want to rely on the dynamic dom/node that's an electric dynamic, which can only be read in electric. At which point we get
(e/defn Hovered? []
(->> (mx/mix
(e/listen> node "mouseenter" (constantly true))
(e/listen> node "mouseleave" (constantly false)))
(m/reductions {} false)
(m/relieve {})
new))
#2023-10-0208:29CanHello! I looking at an example specified about table's in Electric. I need to learn how to sort by table cell and how to implement the pagination feature. Does anyone have a code snipped or a basic example that I can study on it?
Thanks!#2023-10-0208:41s3dseSure, here's my current table function. Pagination is still a ToDo. Please be warned: I'm still an electric noob 😉
(e/defn Table [data fields !per-page search? classes &slots]
(let [!search (atom "")
search (e/watch !search)
per-page (e/watch !per-page)
!items (atom (if (empty? search) (take per-page data) (take per-page (find-data search data))))
items (e/watch !items)
!sort-order (atom {:key nil :direction nil})
sort-order (e/watch !sort-order)]
(dom/div (dom/props {:class "flex flex-wrap mx-4 my-3 gap-2 justify-start dark:text-gray-300"})
(when search?
(ui/input search (e/fn [v] (reset! !search v))
(dom/props {:type "search" :placeholder "Type to search"
:class "border dark:border-moon-700 dark:bg-moon-900 rounded-sm px-2 text-sm"})))
(when (:controls &slots)
(e/for [control-fn (:controls &slots)]
(new control-fn))))
(dom/table (dom/props {:class (:table classes)})
(dom/thead (dom/props {:class (:thead classes)})
(e/for [key (keys fields)]
(let [sortable (-> fields key :sortable)
sort-reverse? (= key (:key sort-order))
th-class (str (when sortable "hover:cursor-pointer ") " " (:th-class (key fields)) " " (:th classes))]
(println key)
(dom/th
(dom/props {:class th-class})
(when sortable
(dom/on "click" (e/fn [_]
(let [asc? (= "asc" (:direction sort-order))]
(reset! !items (sort-by key (if (and sort-reverse? asc?) > (e/defn Input []
(let [message "hello"
submit! (e/fn []
(js/alert message) ;; use local scope
(e/server
;; do something
))]
(dom/div
(dom/textarea
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(.preventDefault e)
(submit!)))))
(ui/button
(e/fn []
(submit!))
(set! (.-innerHTML dom/node) send-icon)))))
#2023-10-0400:46weibasically trying to define an e/fn in the let statement (in order to use vars in the local scope). getting this error that I don't understand#2023-10-0400:54weiit works if I move submit! to its own e/defn, but wondering if there's a way to do the above, similar to regular fn#2023-10-0406:01xificurCLooks like you're only missing a new on submit!#2023-10-0406:34weiah that's it, thanks!#2023-10-0400:59weiI'm seeing the message missionary.Cancelled: Subscription cancelled in my app, but everything seems to work fine. Should I be concerned, and how can I pinpoint the cause?#2023-10-0406:03xificurCThe easiest is to comment bisect. You shouldn't see the error#2023-10-0410:13Fabimhow can I reset the client atom without breaking the value for the server?
I am building drag and drop. unfortunately only the first time works. on subsequent drops (after the reset! ) the lead-dragging is not java.lang.long anymore and does not transact (no error shown).
(dom/on "drop" (e/fn [_]
(e/server
(e/discard
(d/transact! !conn
[{:db/id lead-dragging
:lead/stage id}])))
(reset! !lead-dragging nil)))
initialization is:
#?(:cljs (defonce !lead-dragging (atom nil)))
(e/def lead-dragging (e/client (e/watch !lead-dragging)))
#2023-10-0410:20xificurC2 observations you might be unaware of
• (e/server ...) and (reset! ...) run concurrently
• the reset! fires the lead-dragging watch#2023-10-0410:22Fabimgood to know I actually let the lead-dragging (let [lead-dragging lead-dragging] . this should give me the value right or do i deref?#2023-10-0410:22xificurCit's still a cycle#2023-10-0410:23xificurCyou can first try (e/on-unmount #(reset! !lead-dragging nil)), maybe that'll be enough to resolve the cycle#2023-10-0410:24xificurCotherwise yes, derefing breaks the cycle since you no longer reference the watch but the atom#2023-10-0410:25Fabimthe deref does not seem to get a value @!lead-dragging#2023-10-0410:28xificurCif the reset! to nil runs first it will be nil, which is why I suggested to reset on unmount#2023-10-0410:28xificurCalso, is the deref on the client?#2023-10-0410:30Fabimyes its in the surrounding (if-let [lead-dragging @!lead-dragging) but then does not update when I start dragging (set the lead-dragging value)#2023-10-0410:38Fabim@U09FL65DK putting the unmount in the on drop fn worked. thanks for your help.#2023-10-0415:55telekidI have an atom that I'd like to share between my standard ring HTTP routes and electric. As far as I can tell, the only way to accomplish this is to bind the atom to a dynamic var that closes over jetty – there's no way to pass a value as an argument to electric, right? NBD, just curious#2023-10-0419:31xificurCno, there's no way to pass an argument to electric. Can you share some code/snippet or more concrete information?#2023-10-0420:36Geoffrey GaillardIn electric e/*http-request* is bound to the WS UPGRADE ring request. You could write a ring middleware to assoc your atom in the request map before electric starts.#2023-10-0420:50telekid> You could write a ring middleware to assoc your atom in the request map before electric starts
Ah, of course. That would do, thanks.#2023-10-0512:13pezHere’s a way to detect key presses, and stop detecting them when the app loses focus. It emits a key state if the currently pressed keys, but if some other tab or window becomes active or something like find-in-page happens, the key state will be cleared. Another thing handled is that you can tell the flow to preventDefault on some particular keys.
NB: It doesn’t handle the case that you might want to register a key that is pressed when the app gains focus.
Thoughts?#2023-10-0518:41sergey.shvetsHi, first time trying Electric, and I got stuck on building a recursive tree in UI. Is there a way to do a forward declaration for e/defn? Similar to declare in Clojure?#2023-10-0518:49sergey.shvetsOh, nevermind
e/def + binding work, I just forgot to call it with new.#2023-10-0518:50Dustin Getzbest workaround is probably letfn#2023-10-0518:50Dustin Getzyeah, binding a lambda into dynamic scope also works#2023-10-0518:51sergey.shvetsThanks!#2023-10-0521:44sergey.shvetsDo I have to use e/fn in dom handlers? I'm trying to handle drop event and it has a very noticeable delay (about 0.5 seconds) before firing? My assumption is that the delay is due to some propagation of reactive events through graph? Ideally, I'd like to skip the whole electric graph for this handler, as it is a pure UI handler.#2023-10-0522:23Dustin Getzdom/on! is idiomatic, not sure what’s up with the delay, that is unexpected would need to see more context#2023-10-0522:23Dustin Getzdom/on! takes a clojure fn#2023-10-0523:35sergey.shvetswith all the drag events as dom/on! things work as expected.
I think I pinned down the problem. "dragover" handler with e/fn probably fired too many events and it was processing it. As soon as I switched that to e/on! delay disappeared. dragover fires non-stop (at least every animation frame) while dragging, so it makes sense that it may create a long queue of events to propagate through graph.#2023-10-0523:36sergey.shvetsBut nice to know about e/on! I'll use it when I don't need reactivity#2023-10-0523:36sergey.shvetsThanks for the quick response!#2023-10-0600:03Dustin Getzhmm maybe, we relieve backpressure in these event apis so that shouldn’t be the issue, however the event apis that take e/fn have FSMs inside them for dealing with possibility of latency they are all super complicated so i expect it’s an interaction with that. This is all getting paved but not until next spring at this point#2023-10-0600:11sergey.shvetsMakes sense. Glad there is a workaround for now!#2023-10-0603:45FabimI’m experiencing the same delay for on drop.
the drag over dom/on "drop" areas seems to go in slow motion, because I use dom/on "dragenter"and e/on-unmount in "drop" to update the dragging states in atoms. on! did not fix it.
The UI feels laggy while dragging#2023-10-0604:20sergey.shvetsProbably try to debounce some of the events?#2023-10-0604:44Fabim@U4EFBUCUE you mean using debounce of missionary?#2023-10-0605:05sergey.shvetsIf I understand correct what happening, some events spams a bunch of events and then delay is happening while those events propagate through dag over and over again. If you reduce number of those events it should help reduce delay.#2023-10-0605:07Fabimhow would I reduce the events?
I need to update the drag state so that ui and logic can adapt to the change#2023-10-0605:09sergey.shvetsIs it possible for you to put logic into only dragstart/drop? Those fire once.
I have some logic in dragenter/dragleave but that only setting some classes on DOM so I can move it easily into dom/on!#2023-10-0605:11Fabim(dom/on "dragover" (e/fn [e] (.preventDefault e)))
(dom/on! "dragenter" (fn [_] (reset! !stage-dragged-to id)))
(dom/on "drop" (e/fn [_]
(when (= stage-dragged-to id)
(e/server
(e/discard
(d/transact! !conn
[{:db/id lead-dragging
:lead/stage id}]))))
(e/on-unmount (fn []
(reset! !lead-dragging nil)
(reset! !stage-dragged-to nil)))))
#2023-10-0605:13Fabimthe state is used so switch css classes as well, I could add those using js but then its less declarative.#2023-10-0605:15sergey.shvetsI suggest you get target where it dragged within the drop event. I have "droppable" class on allowable target and then do the following in drop:
(dom/on! "drop" (fn [evt]
(.stopPropagation evt)
(js/console.log "Handle Drop" evt "tree-state" tree-state)
(let [target (.closest (gobj/get evt "target") ".droppable")
block-id (.getAttribute target "data-block-id")
block-type (:dragged tree-state)]
#?(:cljs (do (trigger-tree-command target
{:block-id block-id
:command :add-item
:block-type block-type})
(trigger-tree-command target
{:command :set-dragged
:target nil}
))))
false))
#2023-10-0605:15sergey.shvetslook at the first let binding.#2023-10-0605:16sergey.shvetsjust find a closest droppable to where it dropped.#2023-10-0605:17sergey.shvetsThis way you can drop dragenter logic and I hope remove the delay#2023-10-0605:21Fabimif I remove the dragenter the UI can’t update while the user is dragging#2023-10-0605:23Fabimhow does your trigger-tree-command look like?#2023-10-0605:25sergey.shvetsTry to make dragover a dom/on! command and return false from it after preventing default.
Trigger-tree-command is just firing a custom js event.
#?(:cljs (defn trigger-tree-command
[el data]
(let [command (js/CustomEvent. "treeCommand" #js {"detail" data "bubbles" true})]
(.dispatchEvent el command))))#2023-10-0605:26sergey.shvetsI capture it within the root div to localize all state updates.
(dom/on "treeCommand" (e/fn [evt]
(let [data (gobj/get evt "detail")]
(case (:command data)
:toggle-opened (swap! !state update :opened (fn [opened]
(if (contains? opened (:block-id data))
(disj opened (:block-id data))
(conj opened (:block-id data)))))
:set-dragged (swap! !state assoc :dragged (:target data))
:add-item (e/server
(page-model/block-command (merge {:builder/page-id page-id} e/*http-request*)
{:command :add-item
:block-id (:block-id data)
:item-type (:block-type data)}))))))#2023-10-0605:30Fabiminteresting. rebuilding reframe haha
if you have one !state atom wouldn’t e/watch on !state signal everytime anything in the state changes? how do you listen for only parts of the !state on the client#2023-10-0605:40sergey.shvets!state is a client-side only atom and I guess it's updates every time something changes, but it's not that often. My whole Electric implementation is for one very interactive form, not for the whole application.#2023-10-0605:42sergey.shvetsI also believe that even when it signals it won't update parts of the graph that haven't changed, but I'm not 100% sure about this.#2023-10-0605:43sergey.shvetsIt's my first day with Electric, so take everything I say with a grain of salt :))#2023-10-0610:02Dustin Getzsend us a working gist or repo that is slow and we will make it fast for you#2023-10-0610:24FabimThanks. DMed#2023-10-0610:50xificurC> if you have one !state atom wouldn’t e/watch on !state signal everytime anything in the state changes?
yes, the !state watch will trigger. But it might be less costly than expected. E.g. in (expensive (:bar state))
• there's an (swap! !state update :foo inc) somewhere in the app
• the state watch fires
• state changed, so (:bar state) re-runs
• (:bar state) returns the same value
• (expensive (:bar state)) doesn't re-run because the arguments didn't change#2023-10-0610:53xificurChow do you listen for only parts of the !state on the client1 atom should have 1 e/watch to prevent FRP glitches. Passing part of the state means selecting a part of the watched value
(let [!state (atom {}), state (e/watch !state)]
(Foo (:foo state)))#2023-10-0610:57Fabimokay will unify the atoms into one !state atom then to avoid multiple rerenders.#2023-10-0610:59Fabimthe (dom/on! "dragenter" (fn [_] (reset! !stage-dragged-to id))) still causes the multiple changes causing the ui to lag behind if I drag over multiple elements. would need a way do discard reset! if a new one is triggered#2023-10-0611:00xificurCwe don't advertise any state management solution. Single atom, many atoms, datascript db, rama... All work and have their tradeoffs. I'm just pointing out a global atom is a viable option in many scenarios so if it is one's preference one is fine to try it out#2023-10-0611:43Fabimmaybe there is an easy way to use missionary to queue up client state updates and only signal updates to the e/watch when no other state update is queued#2023-10-0611:46Dustin Getzi still suspect something wrong in userland, dom/on! relieves backpressure, it is not slow#2023-10-0615:29sergey.shvets@U010L3S1XHS can you try wrap your state reset! in some simple debouncer? You can write one in pure js with setTimeout/clearTimeout. While it won't solve the delay UI problem, it will at least help figure out if that's related to Electric or something else. With debouncer all of your handlers should fire immediately once you timeout interval passes, if the problem was too many events scheduled by dragenter/dragover. Put something like 16ms. Or even simpler comment those handlers and see if drop fires immediately and then return them one by one to see where the problem is.#2023-10-0906:03Fabim@U4EFBUCUE thanks for the suggestion. even when removing on! dragenter the on drop still is delayed for a second. not sure why because there are no state changes before.#2023-10-0906:40xificurCwe haven't seen a case where electric would lag behind one second. If there's a clear repro we can take a quick look#2023-10-0610:59Fabimthe (dom/on! "dragenter" (fn [_] (reset! !stage-dragged-to id))) still causes the multiple changes causing the ui to lag behind if I drag over multiple elements. would need a way do discard reset! if a new one is triggered#2023-10-0618:55nivekuilcan an electric app be split across modules?#2023-10-0618:56Dustin Getzwe have not thought about module splitting yet, but after the current incremental compiler workstream lands this will be closer at least#2023-10-0621:00JAtkinshttps://github.com/JJ-Atkinson/electric-data-viewer
I'm not entirely sure why this is borked. I'm inferring that e/hook is cooking up some failures, due to moving the dom/text nodes around. Not sure why they are being moved though. The goal with this code is to make a server-walked viewer for data, so java objects can be arbitrarily inspected.#2023-10-0621:01JAtkinsShould be a relatively small proj - it's based on electric starter app. LMK if I need to trim it down to an MVP or if this is close enough to minimal to be understood#2023-10-0621:10Dustin Getzrepo is private#2023-10-0621:10JAtkinspublic now#2023-10-0621:11Dustin Getzty, what is mount-watcher ?#2023-10-0621:12JAtkinsI needed something to wait for dom elements of an id to mount#2023-10-0621:12Dustin Getzor rather what are you using mutationobserver for#2023-10-0621:12JAtkinsthe viewers don't know anything about children, they just mount dom objects of ::view-options/id id and let children hang off that id by binding dom/with#2023-10-0621:13JAtkinsvery mutate-y heavy UI#2023-10-0621:13JAtkinsif there's a better way for this I'm all ears though#2023-10-0621:23JAtkins#2023-10-0621:23JAtkinsat t=0 the table is rendered, and the dom elements for rendering A and B are attached. t=1 those two elements are seen and children for viewers are mounted#2023-10-0621:38Dustin Getzoh, without looking (on mobile) rebinding dom/node is undefined behavior in electric v2. this is supported in v3 which isn’t out. Because obviously it’s surprising for that to be UB and it’s a perfectly reasonable thing to want to do and quite powerful. In the meantime can this be expressed without rebinding dom/node?#2023-10-0621:40JAtkinsMaybe. I'll give it a try. Sorry didn't know dom/with isn't supported 😬 - guess it makes sense that that's borked behavior#2023-10-0621:41JAtkinsMight be able to get it into a tree of e/fn calls#2023-10-0621:41Dustin Getzyeah i now see you’re calling dom/with not rebinding dom/node, i need to go oook at the src when back at my desk later#2023-10-0622:11JAtkinsgot it to work basically in place with an atom of anonomized e/fn [] atoms. Pushed if you are interested.#2023-10-0622:55Dustin Getzi will try to look, good job figuring that out#2023-10-0623:28nivekuili've had that exact same error rebinding node to js/document.body. workaround was to change render order for some reason#2023-10-0623:32nivekuilyou shouldn't have multiple watches on the same atom I think? view-db/!id->view-fn#2023-10-0623:41Dustin Getzrebinding?#2023-10-0623:43nivekuil#2023-10-0623:11VincentWho wants to build a music streaming service with electric and me 😄
I think Electric is a great fit for such a project
It's such a huge undertaking;
I was wondering: is there a clean way to stream audio via the websocket connection? That's my biggest/fuzziest/vaguest spot right now.#2023-10-0623:38Dustin Getzi would use a raw websocket for something like that today, use electric for the UI parts (you can have N websockets)#2023-10-0623:45Vincentand not sure how i would do gapless playback, that would be a must#2023-10-0623:45Vincentbut maybe socket / channel logic is enough to do that in clj land#2023-10-0623:53nivekuillook at webtransport if you want to do something novel#2023-10-0802:17Vincentdoes webtransport save me from having to make an iOS and android app? 😅
looks like HTTP3 supersocket basically
main use case is mobile users so i ... wonder...#2023-10-0803:09nivekuilmaybe, though I think you'd have to build a clojure webtransport stack first#2023-10-0803:36Vincentsomeone will#2023-10-0710:56Garrett HopperInteresting article that seems to mirror many of the comments made regarding functional effect systems from https://clojurians.slack.com/archives/C7Q9GSHFV/p1692205629281279?thread_ts=1692205629.281279&cid=C7Q9GSHFV
https://www.1a-insec.net/blog/19-reactive-signal-and-build-system/
Particularly the end where the author is looking for the name of the algorithm
Also
> I feel that this algorithm may be adapted to work across multiple machines.#2023-10-0723:35telekidYeah I've since had the thought that it would be cool to be able to persist a expression / node to disk via some kind of protocol#2023-10-0723:35telekidI think this is how you might build a database out of electric? Again more unrigorous shower thoughts ¯\(ツ)/¯#2023-10-0712:57nakkayaWhat would be the idiomatic way to structure the following workflow,
• User uploads a file from electric to a ring handler via POST
• Handler puts the file into a queue for processing
• It takes around 5 to 10 seconds to process the file
• Result is saved to postgresql db
How would I go about notifying electric when pipeline is done processing so I can update the results div. Previously for a similar problem on another project I kept an atom on the backend and watched that from electric for changes but this seems like it is not scalable since an update will cause all clients to redraw their div including the ones that did not initiated computation.#2023-10-0713:27Dustin Getzperhaps the client attaches a uuid to the POST request, and your post handler carries the ID along and puts the ID in the global atom at the end, the client watches the global atom for their ID#2023-10-0713:27Dustin Getzthe atom can contain a set of IDs and the client can disj their ID from the set when done#2023-10-0713:28Dustin Getzyou can also use a missionary discrete flow instead of an atom if you want to get advanced#2023-10-0713:34nakkaya> perhaps the client attaches a uuid to the POST request, and your post handler carries the ID along and puts the ID in the global atom at the end, the client watches the global atom for their ID
I already carry a UUID on the POST request but with this scheme when the atom is updated would this not trigger a update to all clients watching the atom? causing all clients to redraw their components instead of just one. Am I missunderstanding updates or can a client watch for partial changes to an atom? i.e changes only for the UUID?#2023-10-0713:34Dustin Getzok, by "client" we really mean the connected client's backend, so the e/server side#2023-10-0713:35Dustin Getzwatch the atom from e/server and check if the id is in the set from e/server#2023-10-0713:39nakkayaby client I mean each distinct browser session, so 10 users with 10 browsers all watching the same atom (defined on the backend clj side not in a electric file.) I watch this atom for changes from electric. When atom updates would this not notify all 10 browser session to update themself but only 1 of these will require a change?#2023-10-0713:42Dustin Getzsomething like this:#2023-10-0713:43Dustin Getz#2023-10-0713:56nakkayathanks I'll set this up, just to make sure I got the mechanics of this right, (if ((e/watch !bus) id)...) latest value of !bus still gets transferred to all browsers/sessions right?#2023-10-0714:14Dustin Getzno#2023-10-0714:14Dustin Getzuse println to try it out and confirm#2023-10-0714:15nakkayafantastic thanks. 👍#2023-10-0808:34weiasking to get a better understanding.. it doesn't get sent to all clients because of this bit that filters by id in the e/server block, right?
((e/watch !bus) id)
as opposed to the chat example where no additional filter is applied to the watch?
(e/def msgs (e/server (pad 10 nil (e/watch !msgs))))
#2023-10-0719:02Vincenton X (was twitter) I see ads for "Balsa" and how i can make realistic prototypes, then I giggle to myself because my electric app is very close to that but gives me a real result when i'm done! :D#2023-10-0803:06nivekuiltyping animation; need differential electric to do this better?
(e/for [c (->> (y/ap (let [c (y/?> (y/seed elt))]
(y/? (y/sleep 50 c))))
(y/reductions (fn [acc n] (conj acc n)) [])
new)]
(ed/text c))#2023-10-0813:21Dustin Getznot immediately.l clear what this is can you post more context pls#2023-10-0821:01Dustin Getzalso i don’t think m/ap and the other missionary coroutine DSLs are electric compatible yet due to use of mutable arrays in the cloroutine macroexpansion, you’ll need to wrap them in clojure. i believe the symptom of this is they will crash so perhaps you are getting lucky here#2023-10-0821:11nivekuilgood to know. this is just for typing out a string char by char. the only small problem I see with this is that you have to conj into this vector to return from the flow, which you just iterate over for effects anyway with e/for#2023-10-0821:24Dustin Getzyes that seems right#2023-10-0906:45xificurCtyping animation; need differential electric to do this better?if the characters are coming in one by one, in the differential world you could send diffs. But for a small string/collection you shouldn't notice performance issues or delay today.
If you already have the characters then animating them sounds like a job for CSS#2023-10-0907:48nivekuilwould if I could, but I need more than dom. I'm aware how much slower it is to do it through missionary, my finger got tired scrolling the flamegraph#2023-10-0910:35Dustin Getzoh was this a perf question? your typing indicator snippet is not fast enough in some way? please provide context #2023-10-0910:43nivekuilno, this is just a best practices question with no practical perf requirement. I do have perf problems with electric (~500ms to render a component) though I haven't done anything to optimize it aside from glancing at the hilarious flamegraph#2023-10-0910:43nivekuilit's not unexpected that stuff feels jankier than react since it does all the work eagerly, no scheduler or anything#2023-10-0910:52xificurCthere's most likely low hanging fruit inside electric to improve performance, since we haven't done much profiling and optimizations, only the ones we needed to unblock us or client projects#2023-10-0910:54nivekuilyeah I don't see it as a priority, but I was expecting to see layout thrashing and I think it was more gc pressure#2023-10-0814:47HendrikI have a macro, that returns a e/defn , but I can’t get it to work. Here is my minimum failing example:
; in file foo
(defmacro defn2 [label args & body]
`(e/defn ~label ~args ~@body))
; in file bar
(defn2 Foo []
(dom/div (dom/text "meh")))
(e/defn Root []
(new Foo))
This fails with
; Encountered error when macroexpanding hyperfiddle.electric/boot.
; Unable to resolve symbol: Foo
; {:file xxxx, :line 41, :column 4, :end-line 41, :end-column 13, :in [(new Foo)]}
; ExceptionInfo: Unable to resolve symbol: Foo
Am I missing something here?#2023-10-0816:01Dustin Getzdid you remember to :require-macros in cljs#2023-10-0816:41Hendrik🙈 that did the trick. I thought I tested that, but probably I had some stale state in my repl. Thanks for the hint. You saved my nerves :D#2023-10-0911:18CanHello, When I add this code block to my project then I get stale output error when I delete that part code the project works. It is strange because I used the same code in my other project this code is working without a problem I just added in this project extra datomic db here. What am I missing?
(def columns
#js
[#js {:name "Title", :selector (fn [row] (.-title row))}
#js {:name "Year", :selector (fn [row] (.-year row))}])
;If you want to take a look that is my page's gist: https://gist.github.com/Bariscanates404/0813204149ba8b5a3c4e3fac46d75963#2023-10-0911:34xificurCI guess this is a cljc file and the clj part failed to compile. Try wrapping the js-specific data in a #?(:cljs reader conditional#2023-10-0911:53CanI tried restarting project
I tried wrapping in cljc block
Still same 😞.#2023-10-0912:06Dustin Getzas peter says #js reader literal only valid in cljs (also valid in e/client)#2023-10-0912:07Dustin Getzyou also use it in ‘(def data)’ not just columns#2023-10-0915:49Cansorry I sent message on channel. Thanks for the help.#2023-10-0918:36Vincentresolved?#2023-10-0918:38CanResolved, thanks.#2023-10-0920:28nivekuilin general what does it mean when you get can't process event - consumer is not ready when you call ! in observe for the initial value? the consumer I assume is the electric code that calls new on the flow returned by`observe` , but that should happen after observe is run, so the initial ! shouldn't do this since there are no consumers yet?#2023-10-0920:30Dustin Getzit could be a backpressure issue, do you need m/relieve ?#2023-10-0920:32Dustin Getzi.e. the flow has notified but electric has not pulled the value through yet, and another event has appeared#2023-10-0920:33nivekuilah, very possible#2023-10-0920:35nivekuilso that's what relieve is for 🙂#2023-10-0920:43Dustin Getzcrash course in m/relieve:
(m/relieve {} >x) – {} will drop the stale event on the floor, replacing it with new event
(m/relieve conj >x) – [i think that type checks? i'm worried about the initial empty value] conj will build up a vector of events (i.e. a queue) for you to deal with it upstream and now it is your fault when you run out of memory
sometimes you'll chain with (m/reductions) so you can have a discrete reducer stage with a reducing function that is guaranteed to see every event, and then m/relieve after that.#2023-10-0920:44Dustin Getzhere's a real world example of that#2023-10-0920:44Dustin Getzthat will return a set of presently depressed keys#2023-10-0920:46nivekuilconj isn't associative though, even if there were an initial []#2023-10-0920:46Dustin Getzits a reducing fn, i.e. (reduce conj [] [1 2 3])#2023-10-0920:47Dustin Getz(reduce conj [] [1 2 3]) := [1 2 3]
(reduce {} nil [1 2 3]) := 3#2023-10-0920:48Dustin GetzI think there is an initial value issue, i think you need to use m/reductions with conj to provide the initial value prior to the relieve#2023-10-0920:52nivekuilThe set of transferred values must form a semigroup with given function `sg` as the internal
binary operation, i.e. `sg` must be associative
maybe i'm dumb but I thought {} is associative because it always returns the rightmost value no matter what order it's applied, that can't be case for conj unless it's a set?#2023-10-0920:52Dustin Getzwhat docstring is that#2023-10-0920:53nivekuilrelieve#2023-10-0920:55Dustin Getzsemigroup is monoid but without the initial value, i.e. a reducing function as I understand it#2023-10-0920:55Dustin Getzthis is where you'd pass the diff operator for differential signals – you want the diffs to squash inside m/relieve as they build up#2023-10-0920:56nivekuilisn't the idea for the associativity constraint that diffs can come in any order, like crdts#2023-10-0920:56Dustin Getz~i don't understand "sg must be associative" and it actually doesn't compile in my mind~#2023-10-0920:57Dustin Getzi think he means the semigroup's binary operator must be associative#2023-10-0920:58Dustin Getzhttps://en.wikipedia.org/wiki/Semigroup#2023-10-0920:58nivekuilassociative wrt to the collection, which conj is for clojure.core/sets?#2023-10-0921:02Dustin GetzYeah I must be mistaken, conj needs to be applied at the reductions stage before the relieve#2023-10-0921:04Dustin GetzI don't actually understand why m/relieve cannot take a reducing function, and why conj has to be done at a prior stage#2023-10-0921:04nivekuilI think the contract changed now that relieve is a differential primitive#2023-10-0921:04nivekuilhttps://github.com/leonoel/missionary/commit/187ff9eb630b4281d4b1eb1d33f165a0aa5535cf#diff-28432b48e6d85d5914dffdd083192ab48cc46ee44b05297b04039722276dcd98L642#2023-10-0921:06Dustin Getzhm maybe, I dont understand the new associative requirement#2023-10-0921:06Dustin Getzunclear if it's a new requirement or just the docstring was strengthened#2023-10-0921:06nivekuilwell he literally renamed rf to sg#2023-10-0921:06Dustin Getzi understood the change to be backwards compatible#2023-10-0921:09Dustin Getzi.e. i suspect it was an old requirement and the old docstring may have been unrigorous#2023-10-0922:49VincentMagma is math?#2023-10-1006:00leonoel> i suspect it was an old requirement and the old docstring may have been unrigorous
yes, that#2023-10-1006:01leonoelalso conj is not associative, but into is#2023-10-1006:43nivekuilwow into even comes with brackets (into [1][2 3]) (into [1 2][3]) . I was struggling to figure out how to even denote associativity with prefix notation#2023-10-1006:54leonoel> isn't the idea for the associativity constraint that diffs can come in any order, like crdts
that would be commutativity#2023-10-1010:20Dustin Getz@U797MAJ8M yes i was struggling too. Leo what is the conceptual reason for m/relieve not taking a RF?#2023-10-1011:01leonoelit's not a reduction#2023-10-1011:21Dustin Getzwhy not#2023-10-1011:21Dustin Getzwhy can't it be#2023-10-1012:48leonoelif it's a reduction, when does it start, when does it end, what is the initial value#2023-10-1100:31Dustin Getzregarding using conj as a semigroup operator, it is also invalid because it is not “internal” which is to say the inputs and outputs are not the same type#2023-10-0923:59nivekuilprior art? https://github.com/parenthetical/webflex https://bobkonf.de/2019-summer/slides/adriaan-leijnse.pdf#2023-10-1000:34Dustin Getzha we found him in literature review (i think? i forget exactly). looks like he’s put more work into it, there was just the 2019 talk when i saw him in 2020#2023-10-1000:35Dustin Getzhe probably found motivation by seeing our blog post in 2021#2023-10-1008:06FabimI followed the starter app deploy on fly. The health check fails with insufficient memory available. Is
fly scale vm shared-cpu-4x
still enough to run electric?#2023-10-1008:47xificurCour starter app runs on shared-cpu-1x@256MB. Our examples app (https://electric.hyperfiddle.net/) runs on shared-8x-cpu@2048MB#2023-10-1008:49FabimThanks for the information.
my app is based on the `electric-starter-app` #2023-10-1010:15Dustin Getzare you using a database? you may need to tune memory settings for your database#2023-10-1010:16Dustin Getzram is basically free i would just add more ram#2023-10-1010:23FabimNo database #2023-10-1013:11Dustin GetzAlso please post the full error with context? It could be a Fly issue, we hit random Fly issues often, search the forum. Fly issues can even be region specific#2023-10-1013:21FabimGot it to work by deploying a couple of times #2023-10-1015:22Fabimfly seems to be flaky as you said. I redeployed and reached the max restart count. Will experiment with the scaling values#2023-10-1016:16Dustin Getzmax restart count implies the app didn't come up and it retried N times#2023-10-1016:17Dustin Getzcheck for exceptions in log#2023-10-1008:13weiis there any documentation for hfql? i can run the demo app and here's what i see. seems like there's more to it but i can't get anything else to show up.#2023-10-1008:50xificurCyou're right that there's more to it, Geoffrey developed a lot of functionality that isn't visible in the basic demo. Unfortunately HFQL is currently parked while we're busy with other work#2023-10-1009:54weimakes sense, thanks!#2023-10-1012:28HendrikSupport for map meta data in e/defn
I think that the following legal defn syntax is not supported by electric:
(e/defn F {:f 2} []
(dom/div (dom/text "foo")))
This fails with this error:
; Encountered error when macroexpanding hyperfiddle.electric/boot.
; Failed to analyse form
; {:in [(hyperfiddle.electric/fn F {:f 2} [] (dom/div (dom/text "foo")))]}
; ExceptionInfo: Failed to analyse form
; hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:706)
; hyperfiddle.electric.impl.compiler/analyze-form (compiler.clj:690)
; hyperfiddle.electric.impl.compiler/eval23168/analyze--23174/fn--23179/fn--23180 (compiler.clj:759)
; hyperfiddle.electric.impl.compil ....
Should this be supported with electric?#2023-10-1012:40Geoffrey GaillardIt’s true we forgot to implement it. We will add it.
Thank you for the report.
In the meantime this works:
(e/defn ^{:f 2} F [] …)#2023-10-1013:05Dustin Getzgenerally speaking be careful with metadata and electric clojure, we do not transfer metadata and it is not checked by clojure.core/= which is used by the reactive propagation engine to decide if a computation can be skipped#2023-10-1013:07Dustin GetzWe considered transferring metadata between client/server, but declined to because it can result in very difficult to debug issue: if the metadata contains an unserializable reference, you'll get an unserializable reference transfer but have no clue to help debug it. Some libraries put weird shit in metadata#2023-10-1013:08Dustin GetzWe propose that there is no valid use case for userland runtime metadata (due to the value equality issue), so let us know if you find one#2023-10-1013:09Dustin GetzPlus it sucks in Clojure anyway, every time I have tried to do something interesting with it the edge cases were too severe#2023-10-1013:12henrikI use metadata often, but mainly :doc, :arglist, :deprecated, etc.#2023-10-1013:21HendrikThanks for your answers. I am currently working on a routing feature. The routing table is statically compiled and to do that I store the needed data as meta data on the var.
I am not done yet. Usage will look like this :
(defrouter Home [primary-email]
{:segment ["home"]}
(dom/div (dom/text (str "Hello " primary-email ", welcome to Aeditto Backend."))))
(defrouter NotFound []
{:segment ["not-found"]}
(dom/div (dom/text "not found")))
(defrouter Settings []
{:segment ["settings"]}
(dom/div (dom/text "settings")))
(defrouter Nested []
{:segment ["nested"]
:targets [(Settings)]}
(dom/div (dom/text "nested"))
)
(defrouter Root [primary-email]
{:segment [""]
:root true
:initial [`Home {}]
:targets
[(Home primary-email)
(Nested)]
:not-found (NotFound)})
;
(e/defn [] (new Root "
Metadata on the vars looks like this:
{::Bar
{:segment ["bar" :id],
:targets []},
::Bar2
{:segment ["bar2"],
:targets []},
::Foo
{:segment ["foo"],
:targets [[::Bar :id "arg"]
[::Bar2 "some arg"]]},
::Stay
{:segment ["stay"],
:targets []},
::NotFound {:segment ["not-found"],
:targets []},
::Root {:root true,
:targets [[::Foo 42]
[::Stay]
[::NotFound]],
:not-found ::NotFound,
:segment [""]}}
Do you think that meta data is the wrong way to do this?#2023-10-1013:25henrik@U023LKF2PQV Interesting thought with an “Electric native” router. Though Reitit works very well with Electric: https://cljdoc.org/d/metosin/reitit/0.7.0-alpha7/doc/frontend/browser-integration, so unless there’s some specific motivation to write your own, you could use it for your routing needs.#2023-10-1013:38HendrikI must admit, that I never took a closer look into reitit frontend integration. Despite the fact that my version uses reitit under the hood 😄 . Usage of my version is very closely to how you would use routing in Fulcro. Including the ability to prevent routing e.g if you want to react to unsaved data etc... I am not sure how I would achieve this with reitit frontend integration. Do you have an example of reitit frontend integration and electric?#2023-10-1014:17henrikYeah, I thought I saw something of Fulcro in there!#2023-10-1014:17henrikI don’t have time to clean this up as much as I should, and it’s a) in an early stage, and b) got domain-specific stuff in there. I’ve cut out lots of stuff, but you’ll get the gist I bet. Throw out the stuff that doesn’t resolve.
The important bit is that router is a reactive var that gives you a stream of route matches.
(ns app.electric.router
(:require
[hyperfiddle.electric :as e]
[reitit.coercion :as coercion]
[reitit.coercion.malli :as malli-coercion]
[reitit.core :as reitit]
[reitit.frontend.easy :as rfe]))
(defn- route-path
[kind]
(str "/" (namespace kind) "/:uuid"))
(def entity-coercion
{:coercion malli-coercion/coercion
:parameters {:path {:uuid uuid?}}})
(def routes
(reitit/router
[["/" home-name]
[(route-path account/id)
(merge entity-coercion
{:name account/id})]
[(route-path atom/id)
(merge entity-coercion
{:name atom/id})]
[(route-path prototype/id)
(merge entity-coercion
{:name prototype/id})]
[(route-path space/id)
(merge entity-coercion
{:name space/id
:handler (fn [{:keys [entity]}]
(reset! context/sidebar-selection-atom entity))})]
["/debugger/:etype/:uuid" {:name :debugger}]]
{:compile coercion/compile-request-coercers}))
(e/def router
"Returns a flow of route matches, for subscription in Electric."
(->> (m/observe
(fn start!
[!]
(rfe/start! routes (fn route!
[{:keys [data] :as match} _history]
(let [match (add-entity match)
handler (get data :handler helpers/nil-fn)]
;; Run the handler
(handler match)
;; Save ff-params
(reset! ff-query-params
(select-keys (get match :query-params) [:ff]))
;; Return match to stream
(! match)))
{:use-fragment false})))
(m/relieve {})
new))
(defn route-to-entity!
[entity]
(rfe/push-state (->key entity) {:uuid (->value entity)} @ff-query-params))
(defn route-to-root!
[]
(rfe/push-state home-name nil @ff-query-params))#2023-10-1014:20henrikAlmost everything in route! can go away, except for ! match which is what returns successive navigations. (handler match) might be nice to keep if you’re looking to do a side-effect upon navigating to a certain route.#2023-10-1014:23henrikAs you can see, Reitit supports coercions, which you accidentally have an example of in there. If you have stuff that should be coerced to integers, UUIDs, whatever, they are nice.#2023-10-1014:26Dustin Getzfor the record, i think our internal position is that Clojure hanging debug metadata on vars is OK, it's userland metadata that we don't like#2023-10-1014:27Dustin Getzexample : having a http request library return the result object and hanging the http headers as object metas – bad idea, return a tuple#2023-10-1014:27henrikI think there’s some confusion about what’s appropriate metadata vs. just plain data as well.#2023-10-1014:28henrikMy feeling is that you should aim at metadata “not mattering”#2023-10-1014:28Dustin Getzthe point is it will not be seen during equality checks inside the propagation engine, so if you depend on it in any capacity in userland you're gonna be unhappy#2023-10-1014:29henrikNot mattering to equality is almost definitional of metadata, I think.#2023-10-1014:39Hendrik@U06B8J0AJ Thanks for sharing. I think my version could improve from rfe. There are some differences: rout-data [["/" {} ...]] in my code is automatically build at compile time and I have no single top level subscription. There is one for each nested router, which results in a minimal diff of the dom#2023-10-1014:51henrikHow much of your dom changes will have to do with how you choose to react to router matches though. If you have some nested routing logic of [page [sub-page [sub-sub-page]]] or similar, it’s just a matter of how you choose to parse your match.
Return matches where
{:page x
:sub-page y
:sub-sub-page z}
for example, and the app logic can choose to react to those, or not.
Of course, you can (and probably should) abstract somewhat over pure Reitit, but I don’t think it’ll lead to better or worse re-rendering.#2023-10-1014:53henrikThe main point of the Fulcro co-locating this with the components is more of a DX decision than a performance decision as far as I can tell. Not that there isn’t potentially merit to that.#2023-10-1013:14VincentAnyone care to brainstorm SEO strategies for electric app? Am thinking http://prerender.io might be good. Or might jut build a parallel set of components in cljs/rum and then host those at actual endpoints, to be overridden by electric app on page load#2023-10-1019:08VincentMy best theory so far is to make parallel components using tonksy's rum and then embed the electric components into a larger page of rum components. Probably the right way forward.
One thing i'm not sure about is putting more divs into the index.html for the rum asset to override, i reckon it would work as planned.#2023-10-1023:54JAtkinsTheoretically, maybe you can make a clj version of dom that renders to hiccup? #2023-10-1023:54JAtkinsThen just … “just” make e/client mean e/server and compile the code twice. #2023-10-1100:08Vincenthey you're real smart you ought get a raise at your career.#2023-10-1100:08Vincentyes. we need a version of (dom/...) that renders to hiccup or whatever! genius! i guess that is painfully obvious now that i reflect! 😅#2023-10-1100:12JAtkinsI wouldn’t mind a raise lol. Maybe I’ll be able to get one if I make electric sing and dance and make me more productive :)#2023-10-1104:15VincentBasically need to turn (dom/...) into string builder append append append like in tonsky's rum SSR
https://github.com/tonsky/rum/blob/gh-pages/src/rum/server_render.clj
to me it seems like a good approach#2023-10-1107:41Geoffrey GaillardOther idea: instead of rendering directly to the browser DOM, render to an org.w3c.dom instance (jvm), then use the built in methods to get an html string out of it.
Who knows, there might also be a way to get a stream of html chunks to stream to a browser 😉#2023-10-1102:02joshchoI am probably going to sf November 5th to 8th, any couches that I can maybe surf on?#2023-10-1120:33Daniel JompheOur favorite backend platform seems quite testable with RCF’s async testing primitives.
Thanks again, team Hyperfiddle!
(ns op-two-emits
(:require [com.rpl.rama :as r]
[hyperfiddle.rcf :as rcf]))
(rcf/enable!)
(r/deframaop foo [*a]
(:> (inc *a))
(:> (dec *a)))
(rcf/tests
(r/?<-
(foo 5 :> *v)
(rcf/tap *v))
rcf/% := 6
rcf/% := 4)
The example is taken from https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/#Basic_example:~:text=Let%E2%80%99s%20define%20a,invoking%20the%20continuation%E2%80%9D.#2023-10-1120:37Daniel JompheI might be sinning by both:
• aliasing rama’s symbols as r
• aliasing rcf’s symbols as rcf
Especially for Rama, I think team RPL would say it’s overkill when you build your entire system on top of their primitives.#2023-10-1120:37Dustin Getzuse :refer lol#2023-10-1120:39Daniel JompheI love reading rcf/tests but you’re right I should refer at least some of these symbols like RCF’s tap and % at least.#2023-10-1201:09joshchoamazing, maybe there should be like a electric + rama group#2023-10-1212:21Daniel Jomphe#electric-rama might sound nice, but I think it might be great to continue posting about the intersection of these topics here in #C7Q9GSHFV instead of posting into a new channel. Dustin keeps a very close watch on Rama and I don’t think he’d consider this out of topic at all.#2023-10-1212:22Dustin GetzPlease contnue posting here!#2023-10-1212:22Dustin GetzWe have plans to do a Rama/Electric demo as a test case and demonstration of Differential Electric#2023-10-1212:41joshchoWill do!#2023-10-1213:44Daniel JompheI added the Rama Clojure Word Count example.
https://github.com/DanielJomphe/rama-dventures/blob/main/scratch/word_count.clj.#2023-10-1213:55Dustin GetzI dont think TodoMVC is out of reach from this starting point. Starting with the todos-simple demo in the electric starter app#2023-10-1215:01Daniel JompheDecided to add https://github.com/DanielJomphe/rama-dventures/blob/main/scratch/auction.clj. It was a breeze to add, now that I got the hang of the basics.#2023-10-1215:05Daniel JompheI’m annoyed that my current setup is VS Code with Calva and Clj-kondo, which really doesn’t like Rama’s DSL, obviously.
I’m thinking about turning off the annoying linter bits.
I plan to spend a few daily hours copy-pasting a series of examples based on Rama’s initial java-based demos too. Once that’s done, I should start experimenting with the intersection of Rama and Electric ⚡#2023-10-1215:06Daniel Jomphe#2023-10-1215:11Daniel JompheI https://github.com/DanielJomphe/rama-dventures/blob/main/.clj-kondo/config.edn#L10 all those red underlines.#2023-10-1213:44Daniel JompheI added the Rama Clojure Word Count example.
https://github.com/DanielJomphe/rama-dventures/blob/main/scratch/word_count.clj.#2023-10-1215:01Daniel JompheDecided to add https://github.com/DanielJomphe/rama-dventures/blob/main/scratch/auction.clj. It was a breeze to add, now that I got the hang of the basics.#2023-10-1218:54telekidMade a tiny rama session management module (that basically just rips off / simplifies the gallery profile_module ) just to see what it feels like. The DSL's surface area feels huge, but the docs are exhaustive and make learning feel at least somewhat tractable.#2023-10-1218:57telekidgiven that the reactivity primitive https://redplanetlabs.com/clojuredoc/com.rpl.rama.html#var-foreign-proxy calls :callback-fn with "new val" (in addition to diff and old val,) it seems like integration with electric should still be rather performant (if not "fine-grained") even without differential, assuming you aren't querying any really high-cardinality collections#2023-10-1219:06Daniel JompheOh, you did synchronous tests with RCF - I was so much taken by Rama’s reactive workflow that I didn’t realize it’s still completely possible, in many scenarios, to test synchronously. Thanks for helping me realize that!#2023-10-1220:49telekidyeah though tap/% is great for testing reactivity!#2023-10-1313:56Daniel Jomphe@UABU2MMNW do you mind if I add it to my rama-dventures public repo?#2023-10-1314:01Daniel JompheI like to collect many examples for quick reference.#2023-10-1315:13telekidgo for it!#2023-10-1220:34joshchoHow much would it take to build something like this? https://youtu.be/nhTyuuDZe4w?si=qlh-MBq7AebOrCot#2023-10-1220:35joshchoi.e. I will probably build this at some pt maybe ideas?#2023-10-1221:42VincentHow can I have everyone's mouse show in the one interface
like this extension https://webcursors.click#2023-10-1221:43Vincentthinking... record x and y and somehow draw each mouse to the viewport... maybe absolutely positioned div? i guess that would be an easy way 😅#2023-10-1222:15Dustin Getzthere are plug n play js services that take care of it too dunno if better to DIY#2023-10-1301:40joshchoHas anyone done electric + clojuredart?#2023-10-1304:54PanelMissionary will need to be ported to the Dart VM#2023-10-1317:49nivekuilif you have a form like this, the client will run concurrently with the server block:
(e/server (foo))
(e/client (bar))
and if you wanted synchronous behavior you'd nest the e/client. what if you extracted the e/server part out to its own e/defn or macro? is there an idiomatic way to get synchronous behavior still#2023-10-1317:55Dustin Getzwhat do you mean by "synchronous behavior", you mean imperative evaluation order?#2023-10-1317:55nivekuilif server code throws client code doesn't run#2023-10-1317:55Dustin Getzyour desired behavior is that?#2023-10-1317:55nivekuilin some cases#2023-10-1317:56Dustin Getzthe easy answer is write imperative logic in pure clojure#2023-10-1317:57Dustin Getzcan you provide more context as to what problem you are facing that makes you reach for this?#2023-10-1317:59nivekuilsay instead of using the return value on the server directly, it writes to a central state store that gets reactively propagated to various parts of the client#2023-10-1317:59nivekuilnot sure of the exact situation, still working out best practices hence this q#2023-10-1318:01nivekuil(e/server (do-thing))
(e/client (confirm-thing-was-done))
#2023-10-1318:02Dustin Getzi think you need to establish causality, client needs to depend on something#2023-10-1318:03nivekuilmy first impression was, it would be nice to have clojure semantics throughout instead of having to remember that a given expression is colored and will run out of order#2023-10-1318:03nivekuilofc the e/server and e/client stuff makes it explicit but when you start introducing abstraction things get confusing#2023-10-1318:05nivekuilis there an idiomatic way to denote that causality#2023-10-1318:05Dustin Getzyou can compose them, or you can write to an atom and depend on the watch#2023-10-1318:07nivekuilby compose do you just mean doing something with it
(let [_ (server-stuff.)]
(identity _)
(client-stuff))
#2023-10-1318:07nivekuilthat's fine, just wondering if there's anything better#2023-10-1318:08Dustin Getz(client-stuff (server-stuff ...))#2023-10-1318:09nivekuiloh, function composition. but I don't care about the return value#2023-10-1318:10Dustin Getzi need to know what you're trying to do, the answer is likely to not do whatever imperative thing you are doing#2023-10-1318:10Dustin Getzthere is always a functional phrasing#2023-10-1318:11nivekuilprocedural you mean?#2023-10-1318:12Dustin Getzif server-stuff is doing a side effect, you can at least increment a counter and then depend on that counter on the client, a pretend database#2023-10-1318:12Dustin Getzsurely there is a better way than that, but it is a way to model imperative time at least in a functional language#2023-10-1318:17nivekuilI thought this already gives causality with no dependencies on the return value, is this also wrong
(e/server (server-stuff)
(e/client (client-stuff)))
#2023-10-1318:17Dustin Getzi don't think that is causal, it may have undefined behavior that is benefitting you#2023-10-1318:17nivekuilglad I asked then..#2023-10-1318:21Dustin Getzserver-stuff is writing to a database? Can you watch for the next value of the database?#2023-10-1318:22Dustin GetzUI5 has a thing where we track which transactions are pending and have been accepted, so the UI can get feedback as to when the specific transaction that it submitted has been committed, but it was hard to write and it isn't ready yet#2023-10-1318:24nivekuilwait, do network calls come back out of order#2023-10-1318:27Dustin GetzTCP guarantees order but can you clarify the question#2023-10-1318:29Dustin Getzif client issues two concurrent network transfers (ordered in the Electric AST and ordered in the Electric wire protocol), the server will process the transfers in order, but the server blocks are still concurrent so if they sleep for a random amount they are racing#2023-10-1318:30Dustin Getzif the two server blocks, say, write to the database in a worker thread, that's absolutely a race#2023-10-1318:34xificurCa more concrete example would help answering. I don't think there's a general solution, electric is concurrent and introducing causality with some general abstraction will inevitably overdo it#2023-10-1318:41nivekuilthe answer to my question is clear: "no"#2023-10-1318:44nivekuilI feel like your ui5 problem shouldn't be that hard if you had certain ordering guarantees but I guess you want to rewrite it to be differential anyway#2023-10-1318:50xificurCthe answer to my question is clear: "no"not exactly, I just wanted more context. E.g. we've used case to introduce causality in some cases where we deemed necessary. (case (e/server (foo)) (bar)). Here (bar) runs only once the e/server block returns a non-exceptional value#2023-10-1318:52nivekuiloh, well in that case let me revise my conclusion. the answer to my question is case. that's a nice idiom#2023-10-1318:53nivekuilthough i feel like an optimizing compiler might not like it#2023-10-1422:14nivekuilhere's one case where I'm using this idiom: load from server then route
(c/button (e/fn []
(case (-> (load-eql. [(list `session/logout {})])
return-value-effects
merge-result!)
(tx! {::routers {::top-router `Landing}})))
(ed/text "Logout"))
without it, it seems the route code would unmount the button and cancel the merge-result!#2023-10-1522:39nivekuil> i don't think that is causal, it may have undefined behavior that is benefitting you
there is definitely strange stuff going on with the site macros. (e/server (foo.)) behaves differently from (foo.) where
(e/defn foo ([x] ...) ([x y] (e/server ...)))
#2023-10-1600:09Dustin Getzperhaps you are seeing transfer latency, e/fn a currently “sited” i.e. all args transfer to the calling site and also e/fns themselves do not transfer, they are pinned to their site (all this is getting fixed next year, we already worked out most of the design)#2023-10-1600:10Dustin Getzif you post some RCFs or larger gists justifying your approach probably we can find a pure functional phrasing#2023-10-1600:12Dustin Getzfor the record your preformatted snippets come through like this on mobile which is how i do most support - if you upload as an attachment i will be able to read things better #2023-10-1600:13Dustin Getzor screenshot even#2023-10-1600:20nivekuilouch, good to know about the formatting. not expecting support, just giving you guys a heads up. I don't want to break tempo for this, I can easily work around this behavior and as you said you're rewriting a lot of the core stuff soon#2023-10-1600:34Dustin Getzi don’t understand what behavior you’re warning us about#2023-10-1601:07nivekuilthat e/server and e/defn don't compose transparently in some cases but it sounds like that's not news#2023-10-1601:26Dustin Getzthe lambda siting issue is known and straightforward to work around once you know the rules, is that the issue you refer to? or are you referring to concurrent do interacting badly with imperative logic? the latter is won’t-fix, we don’t see how to make electric work at all today with imperative evaluation semantics#2023-10-1601:28nivekuilI was referring to the former, maybe should have made a new thread but I wanted to quote what you wrote earlier. for the latter, case is an acceptable replacement for await (for this one I was asking for support, i.e. the idiom you use)#2023-10-1601:55Dustin Getzok - re that i was just guessing, i don’t have enough info to diagnose what you experienced#2023-10-1321:54Garrett HopperIs hyperfiddle.electric.httpkit-adapter published somewhere, or is it only in the fulcro template?
https://github.com/hyperfiddle/fulcro-electric-template/blob/master/src/main/hyperfiddle/electric/httpkit_adapter.clj#2023-10-1322:21nivekuilhttps://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_httpkit_adapter.clj#2023-10-1322:22Garrett Hopper:man-facepalming: Sorry; I could've sworn I'd tried importing it from the main dependency#2023-10-1513:07VincentQuestion: what is watchable#2023-10-1513:54Dustin Getzon jvm, anything that implements https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/IRef.java#2023-10-1513:54Dustin Getzhttps://github.com/leonoel/missionary/blob/51631ed9f68cb0071041c539acacc4dbfb2206e3/java/missionary/impl/Watch.java#L80#2023-10-1513:55Dustin Getzwhich is the same as calling https://github.com/clojure/clojure/blob/clojure-1.10.1/src/clj/clojure/core.clj#L2144-L2160#2023-10-1513:55Dustin Getzhttps://clojuredocs.org/clojure.core/add-watch#2023-10-1513:57Dustin Getzhere is how one might implement m/watch using clojure.core/add-watch: https://github.com/hyperfiddle/electric/blob/67d05d28cc2d2f686d810ea94f66995f0ca815a9/src-docs/user/missionary/missionary_tutorial_watch.cljc#2023-10-1513:59Dustin GetzGood question thank you#2023-10-1601:31nivekuilis there a list of known issues?#2023-10-1601:46VincentDo I need something special to decipher changes to (*get-in* e/*http-request* [:cookies "username" :value]) in realtime?
(ui/button triggers an ajax (res/set-cookie login from the client... server replies agreeably, but the interface seems to need a page refresh ...? i have a sense that this is not the actual case#2023-10-1601:50nivekuilthe http-request is only made once, when establishing the websocket connection#2023-10-1601:51nivekuilyou can do (set! (.-cookie js/document) "...")#2023-10-1602:34Vincentyou're my hero#2023-10-1717:50joshchoMaybe there's a connection with electric and this visual programming language? Feels highly reactive (albeit on a very different platform).
https://github.com/samuelmtimbo/unit#2023-10-1717:51joshchodemo: https://youtu.be/lvvzolKHt2E?si=Iy6K0VKsgSxbwjQV#2023-10-1717:54Dustin Getzthat's quite something#2023-10-1717:56Dustin Getzill queue the video for couch time sometime, he says its for applications is there todomvc or anything like that? What's the most serious app#2023-10-1721:13joshchoNot sure. I will probably connect with the creator at some point and circle back.#2023-10-1721:14joshchoFor now, I am trying it on my iPad (it seems built for multi-touch actually), and it's really good. Very different trying it out vs video.
https://unit.land#2023-10-1718:51VincentI'm building a tribal link-sharing site in Electric.
I am leaning towards making it open-source.
I have a question about UI interactions and that
if it's a list, and the gravity per item is changing in realtime, certain things will move in realtime in the ordered list.
I need to add "pausing" to this dynamic motion of the ordered list. Any ideas on how I can do this idiomatically in Electric?#2023-10-1719:42nivekuilyou want your program to be a function, so parameterize time#2023-10-1719:43Vincentwow you're smart#2023-10-1719:43Vincentthank you#2023-10-1719:44Vincenti actually don't understand the implications of this statement fully yet 😂#2023-10-1719:44Vincenta slider for playback / playforward !?
could be very interesting...........#2023-10-1801:04Dustin Getzthere is a demo of a timer at the end of http://electric.hyperfiddle.net, careful it’s tricky#2023-10-1719:21VincentCan I use Math/pow on streaming values? 😅#2023-10-1719:43Vincentanswer is Yes#2023-10-1802:38Vincentis it possible to use System/currentTimeMillis?#2023-10-1802:40Vincentimported clj-time.core and System/currentTimeMillis resolves now :man-shrugging:#2023-10-1808:25Geoffrey GaillardIt should work out of the box and importing clj-time.core shouldn’t be necessary. It would be a bug otherwise. Could you confirm? What behavior do you see when trying to call System/currentTimeMillis before importing clj-time.core?#2023-10-1808:30Geoffrey Gaillard#2023-10-1814:13Vincent:man-shrugging: I see "java.lang System is not a known thing"#2023-10-1817:04VincentIt is unclear to me what solved the issue, I think adding clj-time caused missionary to update and maybe that helped. Unclear. If I run into it again I'll take better notes of the console output -- it was quite weird, it was saying Java.lang System not available and I was like huh you don't know what that is? It was in a #?(:clj ...) block ... anyway thanks for asking! apologies i don't have more info#2023-10-1808:17weiThis is my working function that takes an id and atom and creates a slider for a key/value pair in the atom.
(e/defn WeightInput [id a]
(div
(ui/range (get @a id 0)
(e/fn [v]
(swap! a assoc id v)))))
I want to style the slider differently if it has a zero value, so I tried this:
(e/defn WeightInput [id a]
(div
(let [current-val (get @a id 0)]
(ui/range current-val
(e/fn [v]
(swap! a assoc id v))
(props {:class (if (zero? current-val) "accent-neutral-500" "accent-orange-400")})))))
Which doesn't work correctly- for one, the sliders don't move independently anymore. What am I missing?#2023-10-1808:29xificurCinstead of derefing the atom you should be providing a reactive value (e/watch in this case). In this case you need to pass the reader (e/watch'd atom) and the writer (the atom itself)#2023-10-1808:43weithat's brilliant! all working now, thank you#2023-10-1808:48weiI'm trying to use ratios between client and server
(e/defn TestRatio []
(div
(text "server:" (e/server (/ 1 2))
" client: " (/ 1 2))))
gives me:
server:[TaggedValue: ratio, [#object[Transit$TaggedValue [TaggedValue: n, 1]] #object[Transit$TaggedValue [TaggedValue: n, 2]]]] client: 0.5
is there a good way to configure the reader so a clojure ratio is automatically parsed as a cljs ratio? e.g. I'd like to see
server: 0.5 client: 0.5
#2023-10-1810:16Dustin Getzcast to float on the server- iirc this is a subtle clojurescript issue , there might be more details if you search#2023-10-1810:51Geoffrey GaillardClojureScript do not support ratios
https://www.clojurescript.org/about/differences#2023-10-1810:52Geoffrey Gaillard> ClojureScript currently only supports integer and floating point literals that map to JavaScript primitives
> Ratio, BigDecimal, and BigInteger literals are currently not supported#2023-10-1905:56weigotcha thanks#2023-10-1814:49Vincentin other discoveries, i think the interruptedException via xtdb comes from having some properties on xtdb elements that don't exist -- xtdb has some weird issues if you ask for props that don't exist on every entity, i'm thinking this is the root of the bulk of my concurrency issues lol#2023-10-1814:56refsetxtdb has some weird issues if you ask for props that don't exist on every entitywithin where clauses this is generally true - that's just how Datalog works - so if you want optional/`left-join` semantics you can either use pull, run multiple queries, or use or-join (which is effectively a subquery) like https://github.com/xtdb/xtdb/blob/b9d98f5006afb81ae1a161f9918546671b69c71b/test/test/xtdb/query_test.clj#L1957-L1960#2023-10-1817:02VincentI wish asking for things that did not exist just returned :key nil but I suppose it's hard to decisively prove an inexistence? 😅
I have maps with a plethora of keys, I gotta be very careful to not violate an implicate schema it seems. I need an easier way to detect these violations since they creep in over the course of building an app. Is there some level of introspection available to me as an end-user to see what "xtdb thinks" about a particular query or where clause perhaps. Or some in-roads to detecting anomalously shaped data? REPL-driven development plus implicate schema violations throwing a wrench into the engine is programming on extra hard mode in diablo lol.#2023-10-1818:47refsetHmm, I can sympathise but I don't know of any general advice that can help. Liberal use of the or-join approach I linked might be worth experimenting with.
In the future I could imagine some sort of :debug? true option on the query API that could help reason about the execution, showing the counts per clause and the number of false evaluations of predicates in the context of the original query structure. Somewhat similar to https://github.com/xtdb/xtdb/issues/1234
Happy to move the discussion over to #xtdb (apologies for the noise #hyperfiddle 🙂🙇)#2023-10-1819:06nivekuilI find myself reaching for a statechart, @dustingetz any chance you have a "Lambda: the Ultimate Statechart" spiel queued up or am I good to go#2023-10-1819:07Dustin Getzwe use them sometimes, i dont have a strong opinion as of right now#2023-10-1819:07Dustin GetzFSMs i mean, not statecharts#2023-10-1819:08nivekuilmy turn to have an opinion: FSMs suck, too much boilerplate. statecharts are like fsms+behavior trees#2023-10-1819:11Dustin Getzhttps://statecharts.dev/what-is-a-statechart.html#2023-10-1819:11Dustin Getzi guess i dont get the difference, the distinction appears to be that FSMs dont nest? I would have said they nest#2023-10-1819:12Dustin Getzlooking forward to seeing what you come up with#2023-10-1819:13xificurCinteresting read on the topic https://eli.thegreenplace.net/2009/08/29/co-routines-as-an-alternative-to-state-machines/#2023-10-1819:16Dustin Getzhypothesis: FSMs and FRP are isomorphic. Don't know if true#2023-10-1819:18Dustin GetzHere's an example. FSM or cycle + if statement?#2023-10-1819:19nivekuilrust's async/await feature is built on generators which are coroutines, and compiles to a state machine so there must be a relation there#2023-10-1819:20nivekuilnot sure what you mean by fsms nesting, but statecharts can bubble up events so you don't need to duplicate event handlers, and have parallel states#2023-10-1819:17bhurlowWhat’s the current state of the art for whipping up electric apps, are folks primarily forking the starter app?#2023-10-1819:19Dustin GetzI hope so, remember we update it periodically#2023-10-1819:45bhurlowcopy that thanks#2023-10-1902:57VincentI'm recursively rendering reply/comment components and getting StackOverflows, I wonder if there is a way to do it without poppin the stack. Code in thread#2023-10-1902:58Vincent(e/defn ItemView []
(e/server
(let [e (xt/entity db current-item-xt-id)
xt-id (:xt/id e)
item-xt-id xt-id
link (:item/link e)
item-id (:item/id e)
author (:item/minted-by e)
minted-at (:item/minted-at e)]
(e/client
(dom/div (dom/props {:class "itemview fc"})
(dom/div (dom/props {:class "fi"})
(dom/text link))
(dom/div (dom/props {:class "fi"})
(dom/text author))
(dom/div (dom/props {:class "fi"})
(dom/text item-id))
(dom/div (dom/props {:class "fi"})
(dom/text minted-at))
(when (not= current-item-xt-id "")
(dom/div (dom/props {:class "reply-input fi"})
(ReplyCreate. xt-id xt-id)))
(dom/div (dom/props {:class "replies fr"})
(e/server (e/for-by :xt/id [{:keys [xt/id]} (e/offload #(reply-with-descendant-records db item-xt-id item-xt-id))] (ItemNestedReplies. id)))))))))
(e/defn ItemNestedReplies [xt-id]
(e/server
(let [;i (xt/entity db current-item-xt-id)
r (xt/entity db xt-id)
text (:reply/text r)
author (:reply/minted-by r)
minted-at (:reply/minted-at r)
item-xt-id (:reply/item-xt-id r)
upvotes (:reply/upvotes r)
parent (:reply/parent-xt-id r)]
(e/client
(dom/div (dom/props {:class "itemreplies fc"})
(dom/div (dom/props {:class "fc"})
(dom/div (dom/props {:class "fi"})
(dom/text text))
(dom/div (dom/props {:class "fi"})
(dom/text "xt-id: " xt-id))
(dom/div (dom/props {:class "fi"})
(dom/text "item: " item-xt-id))
(dom/div (dom/props {:class "fi"})
(dom/text "parent: " parent))
(dom/div (dom/props {:class "fi"})
(dom/text author))
(dom/div (dom/props {:class "fi"})
(dom/text upvotes))
(dom/div (dom/props {:class "fi"})
(dom/text minted-at))
(dom/div (dom/props {:class "fi"})
(NestedReplyCreate. current-item-xt-id xt-id))
(dom/div (dom/props {:class "fi"})
(dom/div (dom/props {:class "replies fc"})
(e/server (e/for-by :xt/id [{:keys [xt/id]} (e/offload #(reply-with-descendant-records db item-xt-id xt-id))] (e/client (dom/div (dom/text id))))))) ;(ItemNestedReplies. id)))))
(dom/div (dom/props {:class "fi"})
(ui/button (e/fn [] (e/server (e/discard (xt/submit-tx !xtdb [[:xtdb.api/delete xt-id]])))) (dom/text "✗")))))))))
;;item == newsitem
;;parent == parent
(e/defn ReplyCreate [item-xt-id parent-xt-id] (e/client (InputSubmit. "reply" (e/fn [v] (e/server (e/discard (e/offload #(xt/submit-tx !xtdb
[[:xtdb.api/put
{:xt/id (random-uuid)
:reply/text v
:reply/id (nid)
:reply/minted-by online-user
:reply/upvotes 0
:reply/item-xt-id item-xt-id
:reply/parent-xt-id parent-xt-id
:reply/minted-at (System/currentTimeMillis)}]]))))))))
(e/defn NestedReplyCreate [item-xt-id parent-xt-id] (e/client (when (and item-xt-id parent-xt-id) (InputSubmit. "nested reply" (e/fn [v] (e/server (e/discard (e/offload #(xt/submit-tx !xtdb
[[:xtdb.api/put
{:xt/id (random-uuid)
:reply/text v
:reply/id (nid)
:reply/minted-by online-user
:reply/upvotes 0
:reply/item-xt-id item-xt-id
:reply/parent-xt-id parent-xt-id
:reply/minted-at (System/currentTimeMillis)}]])))))))))
#?(:clj
(defn reply-with-descendant-records [db item-xt-id parent-xt-id]
(->> (xt/q db '{:find [(pull ?r [:xt/id :reply/text :reply/item-xt-id :reply/parent-xt-id :reply/upvotes])]
:where [[?r :reply/item-xt-id item-xt-id]
[?r :reply/parent-xt-id parent-xt-id]]
:in [item-xt-id parent-xt-id]} item-xt-id parent-xt-id)
(map first)
vec)))#2023-10-1902:59VincentYou can see the commented out end of the line (e/server (e/for-by :xt/id [{:keys [xt/id]} (e/offload #(reply-with-descendant-records db item-xt-id xt-id))] (e/client (dom/div (dom/text id))))))) ;(ItemNestedReplies. id)))))
And the reason this comment is there is because if I delete from (e/client...) and uncomment that, I get a stackOverflow.
I think i need a way to "bottom out" or "zero out" on the recursive query :thinking_face:#2023-10-1903:33VincentI added another component that does not recurse / recur and it "bottoms out" on 1 nested comment. I could artificially mint several more to add a semblance of depth / nesting. I thought I could do the same with a depth and max-depth#2023-10-1906:34xificurCPlease post code snippets longer than 3 lines as a slack snippet (Ctrl+Shift+Enter or start typing /snippet) or a a github gist or repo#2023-10-1906:36xificurCAlso please include the exception and where you saw it (clj console, browser etc)#2023-10-1906:37xificurCwe've landed fixes on code size and nesting on the main branch, could you retry with the latest git sha?#2023-10-1906:38xificurCdid you check you're not infinitely recursing?#2023-10-1910:28Dustin Getzhow deep is the recursion? (in addition to what Peter asked)#2023-10-1914:05VincentI think it should stop becuase the database result is empty :thinking_face:#2023-10-1914:06Dustin Getzyou need a base case, an if statement guarding the recur point#2023-10-1914:07Dustin Getzdef factorial(n):
if n > 0:
return n * factorial(n - 1)
else: # base case, no recur
return 1#2023-10-1914:10VincentOkay that makes sense. I attempted "let the replies be (e/offload...) if replies render otherwise no render, but that also not working ...#2023-10-1914:11xificurCI think (e/for-by [] ...) will be the base case here, when the query returns an empty resultset#2023-10-1914:12Dustin Getzoh i must have misread#2023-10-1914:12xificurCyou can println to make sure you're not recursing indefinitely#2023-10-1914:51VincentSo in theory, i can check if replies has any entries, and render or not. I think I understand the concept but I am still running into StackOverflows.
If I print replies to console I see [] or [...] results ...
I must be doing something very naively#2023-10-1914:55xificurCplease include the exception you're seeing here in the chat. Have you tried latest electric sha?#2023-10-1915:07Vincentlet me bump up the version ty for reminding 😅#2023-10-1915:19Vincent😅 some stuff broke but unclear why ... investigating. some of my components stopped rendering#2023-10-1915:21VincentHmm.. I had to roll back#2023-10-1915:25VincentMy online list of users, the chat, and some real-time components nested in there were not rendering at all with the bump so I rolled back to v2-alpha-349-ge9996713#2023-10-1915:26Vincentyou are right that the query returns an empty result [] ... not sure how i can prevent rendering once I know that, I think I am missing something fundamental about execution sequence / assignment sequence#2023-10-1916:09xificurCRendering will stop at that point. Just like (mapv foo [])
never runs foo#2023-10-1917:27VincentOkay, I expected that to be true, but I encounter stack overflows, so :man-shrugging:
tap out for now lol
4 levels deep via manual component creation (aka 4 new components) is fine for now and the forseeable future. if i can figure out "infinite replies" on comments that would be awesome, but it overflows so, gotta focus on tubs i can manage the water level in 😅#2023-10-1918:59nivekuilwas trying the ic branch for fun and though incremental is still broken, it seems to be a lot faster (13s->8.5s) with no other changes#2023-10-1919:03xificurCwere you trying to start an application? There's no entrypoint right now#2023-10-1919:07nivekuiloh, I thought boot-client would have been the entrypoint. but just the same old boot seems much faster too#2023-10-1919:11xificurCtoday the client boots and sends electric IR over to the server in the first websocket message. The server interprets the IR. With IC we split the code and the both peers run their compiled part of the program#2023-10-1919:11xificurCthe speedup you saw might just be the fact only half of the code is built#2023-10-1919:12nivekuili'm not using any of the -ic namespaces#2023-10-1919:12xificurCthen I don't know 🙂#2023-10-1919:14nivekuilmaybe some shared code you optimized? anyways thanks 🙂#2023-10-1923:04macrobartfastDoes anyone have a workflow they can share for reusable UI components (like the React crowd often leans toward)?
And as a side question, and less importantly, can anyone share any inspiring app examples, UI'wise?
I understand the magic of Electric is not focused on the UI, but the graphical interface is a practicality I have to deal with, alas.#2023-10-2000:44VincentReusable ui components? I think all components should be reusable if possible, a component for each view is optional, and a component for each item you're going to iterate over...
My workflow is basically:
1. get a working example repo and merge/mold them together if I need 2 features
2. create InputSubmits. for each of the data thingies I need, like posts, or users, or whatever
3. Generate some data via text input and have them show to the ui immediately
4. basically make sure i have this immediate database-like view on all the data before i start drawing up more comlpex components
5. then, start working on the app in terms of interface,
6. in electric land, lines of code currently affets compilation time so... fewer components is better.! 😅
#2023-10-2000:45macrobartfastThanks tremendously for this!#2023-10-2001:00VincentWhen I hit the button to upvote items on my list of ranked items, i get triple, quadruple, quintuple "upvotes" on a single click -- i reckon this is because it's "streaming" the (inc upvotes) command and that actually appllies more than one time. So it makes me wonder if i simply need to be more robust about ensuring every user can vote only once, or if there is some sort of fancy back-pressure swags i don't know how to use yet 😅#2023-10-2001:11Dustin Getzpost your code, we will help#2023-10-2001:33Dustin Getzthis is on the electric ui layer which event hauling idioms aren’t very natural yet, we’re working on a better idiom, for now we have to teach you the tricks#2023-10-2006:16nivekuilwhy does this need the IIFE? can the compiler not plumb reactivity inside ap? I remember there was some problem with it, not sure if this was it#2023-10-2006:42leonoelthis is a missionary issue, ap macroexpansion is not electric-friendly. The coroutine state is emitted as a mutable array and interpreted as electric code, the array does not depend on any reactive value so it's not reconstructed, the same state is incorrectly reused for all ap runs.#2023-10-2006:56nivekuilthanks, sounds like I can't accidentally trigger this behavior outside of this case#2023-10-2008:09nivekuilis this functional programming#2023-10-2008:12xificurC(y/reductions (fn [play? _] (when play? (sound/play ..)) (not play?)))#2023-10-2008:13xificurCalso, I don't understand the question. Are you asking if this is still functional if it's orchestrating side effects?#2023-10-2008:18nivekuiljust thinking about how un-clojurey it feels. you think it's better to emit a boolean than to not emit?#2023-10-2008:23xificurCthe reductions I posted rids you of the eduction. One step less#2023-10-2008:25xificurCclojure doesn't solve IO. Missionary does, in a functional way. y/reductions is a flow (a recipe to orchestrate IO) and flows compose correctly wrt IO#2023-10-2008:26nivekuilah true, thanks. that makes it feel a lot better#2023-10-2009:07nivekuilI think this has to be the idiomatic way in electric
(let [typed (Typing. (:text ctx))
idx (new (y/reductions (fn [acc _] (inc acc)) 0 typed))]
(when (odd? idx) (sound/play sound/DialogueBoop :rate 0.65 :volume 0.6)))
and now you can use idx to know when it's done. not sure if there's a way to know when a flow has terminated generally#2023-10-2009:21xificurCyou could stay in electric completely if you wish to#2023-10-2009:21xificurC#2023-10-2009:24xificurC> not sure if there's a way to know when a flow has terminated generally
You can run a clj fn when the flow is about to unmount with e/on-unmount#2023-10-2010:17Dustin Getzmissionary flows can terminate, what is the requirement? discrete flows terminate a bit more natively than continuous flows though, bc signals broadly are defined for all time#2023-10-2010:23Dustin Getzwell, that may not strictly be true because a child flow (continuous or discrete) can terminate and invoke cancellation effects as peter alluded, but then it can reboot. Cancellation on missionary propagates as an exception (causing siblings and parents to cancel per supervision)#2023-10-2010:24Dustin Getzi think continuous flows always cancel by exception? as opposed to discrete flows which can complete? possibly continuous flows can also complete its just weird and rare, more of an optimization to say “this signal is invariable and the current value is final”#2023-10-2110:07leonoelobserve self-termination is not possible now, but may be in the future if needed. As a workaround you can push a sentinel value and add an eduction stage with take-while.#2023-10-2110:11leonoelThe notifier/terminator are not called because you're calling the flow with new. This is electric specific, missionary is unaware of that#2023-10-2110:13leonoelPassing notifier/terminator explicitly is expert mode, you should not do that unless you're learning internals#2023-10-2110:49nivekuilI figure electric supplies its own notifier/terminator to the flow with new, but I was hoping it would compose with your own instead of just overriding them#2023-10-2115:42Dustin Getznew when called on a missionary flow doesn’t propagate parameters like that#2023-10-2115:44Dustin Getzwhat new does under the hood is bind the arguments in dynamic scope as %0 %1 etc and then joins the flow. an e/fn compiles down to a flow that optionally reads from %0 etc. #2023-10-2115:45Dustin Getzif you want to manually wire notify and terminate you should invoke the flow as a function per the missionary api#2023-10-2115:48Dustin Getzif you want to escape from electric context to intercept notify/terminate at some point in the electric program- i actually don’t know how to do that, the idea confuses me actually #2023-10-2118:50nivekuil(flow n t) => iterator => flow => electric?#2023-10-2119:40nivekuilkeeping references to e/fn across reloads will always crash the server right? example use case is passing arbitrary code to be shown in a modal#2023-10-2119:59Dustin Getzum yea you can't mix and match electric closures across reloads#2023-10-2120:00Dustin Getzwill it always crash the server? IDK, it's undefined behavior in a bad way, if you're lucky it will crash#2023-10-2120:00Dustin Getzwhy do you want to pass code across reloads? I am confused#2023-10-2120:01nivekuilso you don't have to re-open the modal. hot reloads should maintain state#2023-10-2120:01nivekuilI know it crashes it now, was wondering if this is something possible to fix with incremental#2023-10-2120:02Dustin Getzwe intend to support this but it's not a design goal of the current incremental compilation sprint#2023-10-2120:02nivekuilok, thanks#2023-10-2120:03nivekuilso "be able to store electric code as state" is on the roadmap#2023-10-2120:03Dustin Getzto be exact, eval is what's on the roadmap, this will probably be fixed in passing#2023-10-2120:03Dustin Getzi accept the argument that hotcode reload should maintain state#2023-10-2120:05nivekuilah yeah, I need eval too I think. I finally solved the router transition problem (turns out it's isomorphic to blue-green deployments), can't get rid of the ugly (new (case route Component Component ...))` stuff yet though#2023-10-2120:09Dustin Getzyes agree#2023-10-2122:37nivekuiljs/scheduler.postTask works very well with electric. react has zero advantages now I think#2023-10-2122:38nivekuilhttps://developer.mozilla.org/en-US/docs/Web/API/Prioritized_Task_Scheduling_API#2023-10-2200:54VincentYour favorite way for me to share code with y'all?#2023-10-2202:54Dustin Getzslack attachment (readable on mobile) or gist or repo#2023-10-2202:55Dustin Getzodds of us debugging your snippet are dramatically higher if we can repro the issue on our machine in 10 seconds or less#2023-10-2210:10braai engineerI'm seeing some weird drag event behaviour with Electric {:mvn/version "v2-alpha-428-g22937f75"}. As far as I can tell, I did not make any code changes, but drop event doesn't fire and the drag icon disappears shortly after appearing.
Events for dragstart, dragstop, dragenter and dragover all work, but drop doesn't. Here is an exception:
user.cljs:21 Cannot read properties of null (reading 'appendChild')
in ( hyperfiddle.electric-dom2/new-node nil div ) in quey/manage/core.cljc
in case default branch
in (case "0" ...)
in reactive (defn MinimalDragExample [] ...) in quey/manage/core.cljc line 154
in case default branch
in (case 0 ...)
in reactive (defn Dashboard nil ...) in quey/manage/core.cljc line 709
in (try ...)
TypeError: Cannot read properties of null (reading 'appendChild')
at Function.hyperfiddle$electric_dom2$new_node (electric_dom2.cljc:46:22)
at Function.eval [as cljs$core$IFn$_invoke$arity$4] (core.cljs:3947:15)
at Function.eval [as cljs$core$IFn$_invoke$arity$3] (core.cljs:3942:7)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (core.cljs:3936:7)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (core.cljs:3971:7)
at G__37575__delegate (runtime.cljc:99:20)
at G__37575 (runtime.cljc:96:19)
at eval (Latest.cljs:34:42)
at missionary$impl$Latest$transfer (Latest.cljs:44:45)
at Object.eval [as cljs$core$IDeref$_deref$arity$1] (Latest.cljs:10:17)
Here is a minimal example that reproduces in Brave (Chrome) & Firefox inside my app (but not a minimal app):
(e/defn MinimalDragExample []
(e/client
(let [!drop-counter (atom 0)]
(dom/div
(dom/div
(dom/props {:draggable true
:style {:background "blue"}})
(dom/on "dragstart" (e/fn [e] (js/console.warn "dragstart" e)))
(dom/text "Draggable"))
(dom/div
(dom/props {:style {:background "red"}})
(dom/text "Droppable. Dropped " (str (e/watch !drop-counter)) " times.")
(dom/on "dragenter" (e/fn [e]
(js/console.warn "dragenter" e)))
(dom/on "dragleave" (e/fn [e]
(js/console.warn "dragleave" e)))
(dom/on "dragover" (e/fn [e]
(.preventDefault e) ;; allow drop
false))
(dom/on "drop" (e/fn [e]
(js/console.warn "drop" e) ;; this doesn't fire
(.preventDefault e)
(swap! !drop-counter inc)
false)))))))
~Could be another problem in my app that I'm not aware of, so sorry if it's not Electric.~
OK, must be something in my app...#2023-10-2210:50braai engineerHmm, must be something else in my app because it works in another simpler Electric app - so must be something wrong with my setup.
Also downgraded to v349 and same exception in my main app.#2023-10-2211:27braai engineerHmm I've commented pretty much everything out in my app, and still get the appendChild error even for the simplest of components, e.g. (dom/div (dom/text "test")
Any ideas what I'm doing wrong?#2023-10-2212:43Dustin Getzdom/node is nil, is your entrypoint correct?#2023-10-2212:45Dustin Getzor maybe could be a client/server issue, reading dom/node from server, or binding it on the server#2023-10-2219:44Vincentis it necessary to wrap (e/watch..) in (str ..) ? just wondering#2023-10-2221:30Dustin Getzno, dom/text will coerce to str#2023-10-2221:38denikneeded promises in electric and found
https://github.com/hyperfiddle/electric/blob/cc55772f18bc46373f131e092fc20055c8062b59/scratch/dustin/y2022/m/missionary_promise.cljc#L5-L13
and then rewrote it to#2023-10-2221:39denikand it works just as well, why are the closures / absolve needed?#2023-10-2221:54Dustin Getzhttps://gist.github.com/dustingetz/d3fb5338b43ef86df5f97387f1d5e249#file-await_promise-clj#2023-10-2222:08nivekuilthis should be part of electric. why are we spending time digging through github instead of building our apps#2023-10-2222:31denikthanks @U09K620SG to @U797MAJ8M’s point it might be worthwhile to have a contrib/util alpha namespace#2023-10-2222:42Dustin Getzthis is missionary, i think we do have missionary-contrib and possibly also electric-contrib #2023-10-2222:43denikshould this be added maybe?#2023-10-2222:43Dustin Getzpriority is differential electric and incremental compilation, any serious work (getting all the contrib stuff reviewed by Leo for correctness and then polished) comes behind those#2023-10-2222:44Dustin Getzwelcome to add it if it’s not already ther#2023-10-2222:44denikah I did not know contrib falls under the same scrutiny#2023-10-2222:44denikfull impl combining electric and missionary#2023-10-2222:45Dustin Getzit does not, but it still needs correctness review otherwise users will have broken programs that we have to debug#2023-10-2222:48Dustin Getzneeds tests as well#2023-10-2222:48denikokay no worries. I’m fine with maintaining an electric util ns for now#2023-10-2222:49Dustin Getznice send it,i’ll add it to our ticket for consideration #2023-10-2222:50denikdid not push this change yet. Will do so in the next couple days. it’ll be here https://github.com/lumberdev/tesserae/blob/master/src/tesserae/ui/electric_util.cljc#2023-10-2222:51Dustin Getzok#2023-10-2222:51denikhowever, what’s already there might be useful for others, especially for web ui dev#2023-10-2307:21leonoelWhen the promise constructor supports AbortSignal I recommend this pattern, which is closer to task semantics and more correct regarding cleanup
https://gist.github.com/leonoel/ee46ddf2629da45e75f4e411d18030b7#2023-10-2222:31denikare there any examples of webpush in electric?#2023-10-2222:32denik😯 @U2DART3HA looks like you made the webpush lib for clojure. https://github.com/ggeoffrey/webpush-clojure#2023-10-2307:02Geoffrey GaillardYes, I haven't been using it for a while, I don't remember what's its status. Readme says not production ready, but it has been used in prod for a while without issues. It might need to be updated/refreshed a bit. Contribs are welcome :)#2023-10-2307:10Geoffrey GaillardNote Webpush implies Service Workers. Not sure if it makes sense to run an electric program in a Service Worker.#2023-10-2315:43denikI’m planning to use javascript in the service worker for a PWA to unlock background push notifications#2023-10-2315:48denikbut why would it not make sense to have electric run in a service worker?#2023-10-2318:40Geoffrey GaillardService Workers have some serious time/resources constraints that wouldn't play well with a client-server electric setup, if I'm not mistaken.#2023-10-2304:22VincentFound out that on the clientside I see null as a nestedReplies input which is weird, i think the query should result in an empty set not a null id, but it returns an "undefined" somehow (??) still investigating#2023-10-2310:35Dustin Getzlikely unserializable reference transfer, are you watching logs?#2023-10-2310:36Dustin Getzoh, nvm you see “undefined” not nil, that generally means an arity error in js#2023-10-2315:36VincentAh interesting#2023-10-2401:32Vincentmaking the background-color: transparent; on a (ui/button (e/fn [] ...)) seems to have affected performance?#2023-10-2401:51Vincenti acknowledge i sound insane^#2023-10-2401:54nivekuilbrowser optimizations are fragile but the profilers are very good#2023-10-2401:54nivekuilyou probably want to be using opacity anyway#2023-10-2401:57Vincentopacity#2023-10-2402:10nivekuilwhat are you trying to do? you should always animate gpu accelerated properties. this is about the only time you can notice css performance issues#2023-10-2402:11nivekuilor are you trying to do a css reset on buttons? background: none is also somehow slow?#2023-10-2402:12nivekuilthere used to be unfortunate behavior with some properties like box-shadow and scrolling but that stuff hasn't been an issue for a while. in any case it should show up in the profiler#2023-10-2402:31Vincentyeah just messing with button styles#2023-10-2414:44henrikWhere should I bind my transit readers/writers? Neither option below works:
(electric/handle-electric-ws request
(fn [write-msg read-msg]
(binding [io/*write-handlers* coerce/write-handlers
io/*read-handlers* coerce/read-handlers]
(electric/electric-ws-message-handler request
(fn [msg]
(binding [io/*write-handlers* coerce/write-handlers]
(write-msg msg)))
(fn [msg]
(binding [io/*read-handlers* coerce/read-handlers]
(read-msg msg)))))))#2023-10-2414:46henrikUsing the HTTP-Kit adapter, aliased to electric for some reason#2023-10-2414:48xificurCwe did not publish a clean API, for now you can alter the necessary vars as in https://github.com/hyperfiddle/electric/blob/master/src-docs/wip/demo_custom_types.cljc#2023-10-2414:49henrikPerfect, thanks @U09FL65DK#2023-10-2419:51nivekuil#?(:clj
(do (alter-var-root #'
merge write-handlers)
(alter-var-root #'
merge read-handlers))
:cljs
(do (set!
(merge
write-handlers))
(set!
(merge
read-handlers))))
is what I do for reference#2023-10-2416:35grounded_sageI am looking to render a lot of content and digging in deeper to the Datomic browser to understand how to do large datasets with good performance.
Datomic browser has [contrib.gridsheet :as gridsheet :refer [Explorer]] when I look in the Electric source at contrib there is a note to deprecate for HFQL grid which this line says that is used by datomic browser? Though I cannot find these references. Are these closed source?
https://github.com/hyperfiddle/electric/blob/cc55772f18bc46373f131e092fc20055c8062b59/src/contrib/gridsheet.cljc#L2#2023-10-2416:42Dustin Getzit is not deprecated yet#2023-10-2416:42Dustin GetzHFQL grid is not release ready#2023-10-2416:47grounded_sageOh I get it. The note is a reference to what is used by it for updating purposes.👍#2023-10-2417:51grounded_sageGrateful that you's have this even in here. Huge time saver.#2023-10-2500:27grounded_sageI’ve been bouncing in and out of different projects using electric and it’s a sheer delight every time I am working with it. #2023-10-2511:58VincentWorking on a link-sharing site, built w/ Electric Clojure 😄
could use your feedback
http://NextApex.co#2023-10-2512:53Dustin Getzcool#2023-10-2512:54Dustin Getzthe "streamy rendering" in the thread layout page is an issue that differential electric is designed to solve, so stay tuned for that#2023-10-2516:00danielnealWill differential electric be a drop-in replacement?#2023-10-2516:52Dustin Getzyes that is our intent, subject to the usual minor breaking changes of alpha software#2023-10-2516:55danielnealNice!!#2023-10-2513:43telekidWhile we're doing show and tell, if anyone wants a login to https://demo.hypo.sh/ (built with electric,) DM me and I'll send you an invite#2023-10-2513:55xificurCwhat is the app?#2023-10-2514:11telekidA daemon on your laptop records a 24-hour rolling window of stdin / stdout across all of your terminals. When you encounter a problem with your dev environment, you can go back in time, slice out a window of your command history, and submit it to the web UI, which allows others to play back your terminal interactions. Kind of like a multi-term, time traveling version of asciinema with collaborative functionality. Three minute demo video: https://www.youtube.com/watch?v=mGBo8eibTR8#2023-10-2514:12telekid(It's designed to be self-hosted by companies, so there is no way for you to submit recordings to my demo environment, but you can play around with the playback interface.)#2023-10-2514:43mmerI like video. It does seem amazing to me that we are still spending our lives at terminal sessions !#2023-10-2521:53VincentThis is really cool. And veri impressively done especially w/ Electric keep up the good work. Do you have some users already ? I think this would be super useful for teaching teams things via term.#2023-10-2619:20telekidNo users yet, currently arranging my first partnership#2023-10-2619:46Vincentso cool, congrats#2023-10-2609:18grounded_sageI'm missing something here but haven't figured it out yet. Trying to capture the mouse position on a component.
(defn mouse-position> [element]
(m/observe
(fn [!]
(let [sample (fn [e]
(! {:x (.-clientX e) :y (.-clientY e)}))]
(.addEventListener element "mousemove" sample )
#(.removeEventListener element "mousemove" sample)))))
(defn mouse-position< [element]
(->> (mouse-position> element)
(e/throttle 16) ; RAF interval
(m/reductions {} {:x 0 :y 0})
(m/relieve (fn [acc evt] {:x (.-x evt) :y (.-y evt)}))
(m/latest identity)))
Called liked this in a component.
(mouse-position< dom/node)
#2023-10-2609:23xificurCI think m/relieve is wrong, it should be (m/relieve {}). The caller needs to join the missionary flow with new, so the callsite should look like (new (mouse-position< dom/node))#2023-10-2609:29xificurCnote that we already export mousemove events as a singleton https://github.com/hyperfiddle/electric/blob/cc55772f18bc46373f131e092fc20055c8062b59/src/hyperfiddle/electric.cljc#L206#2023-10-2610:14Dustin Getzi believe the one in master is broken#2023-10-2610:26grounded_sageThe (new (mouse-position< dom/node)) and (m/relieve {}) did the trick. Though I'm not getting my crosshair rendering anymore. So have to figure out why that it.#2023-10-2617:27Dustin Getzalso i dont understand why we are throttling, I know i have seen that pattern in the electric repo somewhere but maybe it is an old workaround#2023-10-2617:38grounded_sageIt didn't really seem to do the trick so I am not throttling now.#2023-10-2621:10grounded_sageOh and I got it from @U0ETXRFEW searching this channel#2023-10-2621:15pezI don’t really understand the role of the throttle either. Just that things didn’t work without it. 😃#2023-10-2621:17pezAnd right now I don’t even remember if the latest code for our mouse crosshair has the throttle still in place.#2023-10-2623:15joshchoSince people here may be interested; how do you solve the yak-shaving problem? Essentially yak-shaving is working (too much) on the tools to solve the problem. I tend to err too much on this side, but often end up finding powerful technologies to justify the search/work.#2023-10-2623:33hiredmanI sigh a lot instead of working on tools, think about https://codahale.com//work-is-work/, go get a coke#2023-10-2707:19henrikThe economics are different when working in a team vs. working by yourself. I’ve spent time on conveniences that I probably wouldn’t have bothered with if it only would impact me. But since they impact a certain number of other people, the gains increase.
When Electric solves incremental compilation, it won’t just cut compilation times by <insert number here>, it will cut compilation times by <insert number here> times <number of Electric users>.#2023-10-2707:19henrikIf you’re Google, you could probably measure it on the electricity bill.#2023-10-2712:47Dustin Getz"do things that don't scale"#2023-10-2712:51Dustin Getzhonestly the motivation to ship is to watch retards get rich on things like remix_web while we barely survive for years. As well as learned humility from multi-year efforts to shave a yak that failed in the end because "built wrong thing" 'idea was wrong" "didn't work in practice" etc.#2023-10-2712:51Dustin Getz"Time is the denominator"#2023-10-2712:55henrikElectric, at least, is none of those things#2023-10-2713:13joshchoI think also sth like “have class” is a thing. At least it’s a dimension (other than money)#2023-10-2712:47Dustin GetzQuick update about incremental compilation since I know you're all excited for this: we have the starter app building under the incremental compilation branch, and all unit tests passing. Remaining major milestones are: port key compiler changes from master, test/validate with our customers (the heaviest users with largest projects), then release#2023-10-2721:29nivekuilis it intended that missionary interop in electric defs break now?
evaling (e/def test (let [x 1] (m/ap x))) results in:
WARNING: Use of undeclared Var missionary.core/switch
WARNING: Use of undeclared Var missionary.core/fork
WARNING: Use of undeclared Var missionary.core/park
WARNING: Use of undeclared Var /x
Syntax error macroexpanding cloroutine.core/cr at (src/main/app/electric/components.cljc:177:1).
Could not resolve var: x
#2023-10-2721:32Dustin Getz"now" ? are you on the unreleased IC branch?#2023-10-2721:32Dustin GetzRegardless, m/ap has never been supported in electric context#2023-10-2721:32nivekuilyes, been on it for a while (not using ic till now). I'm aware, so not sure if it's intended or not#2023-10-2721:32Dustin Getzhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1696798901599769?thread_ts=1696734412.766099&cid=C7Q9GSHFV#2023-10-2721:33Dustin GetzIC branch is unsupported as of yet because it is not finished, strongly recommend not doing this#2023-10-2721:34Dustin GetzI am surprised that you find it usable, I believe async stack traces are missing, the entrypoint is missing ...#2023-10-2721:35nivekuilhaven't been using the -ic namespaces, with no code changes the ic branch compiles like 30% faster. it looks like there's entrypoints now though?#2023-10-2721:36Dustin Getzunsupported, i don't know, good luck#2023-10-3019:31nivekuilworks well enough to ship https://nivekuil.itch.io/cachinko but there seems to be a bug inferring bindings through macro, looking forward to the release#2023-10-3019:33Dustin Getzvery cute#2023-10-3100:59joshchosome devtool ideas are only possible realistically with this#2023-10-2721:07denikcreated an electric PWA with notifications!#2023-10-2721:22denikrepo: https://github.com/lumberdev/tesserae
master is broken as this is wrapped in another lib we use on prod. however, working commit is in the readme#2023-10-2722:16grounded_sageNice! I’m in the process of building an app which I want to be a PWA on mobile with notifications. #2023-10-2722:30deniknice! impl is actually not specific to electric feel free to ask Q’s if you have any#2023-10-3017:26Vincenttyvm#2023-10-3021:05bhurlowvery cool#2023-10-2800:42grounded_sageFor tracking a mouse cursor. Would you say there is much difference performance wise between using an atom or missionary? e/watch is what I use now which works fine. But curious if there is something subtle to consider.#2023-10-3017:21VincentI think because the mouse x,y is a clientside value, e/watch will be quite nice#2023-10-3017:24grounded_sageThat’s what I have currently. #2023-10-3009:53grounded_sageI’m not very familiar with what differential dataflow will allow. What kind of implementations are there that becomes an anti-pattern or redundant with this feature? What is something that this makes possible that isn’t wi the out it?#2023-10-3010:15Dustin Getzit’s just faster, it will also make streaming apis more natural to integrate. we did it to remove technical debt from the implementation and fix some perf issues #2023-10-3017:22VincentYeah. Everything is tracked by an id in the for-by which ensures you only get what the difference is -- conforming to this pattern is key. lots of examples in the example applications that you can start branching from#2023-10-3017:24grounded_sageYea I got that one. I was just curious about if I was missing anything. I don’t know what the difference will be for electric even if under the hood. #2023-10-3017:24grounded_sageI’m assuming it does a check and only streams changed values? But I feel like there is more.#2023-10-3017:25Vincentstreaming is an interesting world of faucets and backpressure#2023-10-3101:23Dustin GetzLeo’s next talk - Tues 10/31. about situated programs, what is the central problem, and how missionary solved it#2023-10-3114:04henrikI’m looking for a general way to fake a causal relationship between sibling graph nodes. Here’s my attempt:
(defmacro cause
"Fake a sequential causal relationship over `forms`."
[& forms]
(let [[form & forms] forms]
(if (seq forms)
`(case ~form
(cause ~@forms))
form)))
Is this alright?#2023-10-3114:18Dustin Getz1- Don't macro this, we do not promise to keep this working. Write it out each time
2- we want to understand what circumstance you're encountering that makes you reach for this, so we can solve it in a better way#2023-10-3114:20Dustin GetzSearch "causal" for past discussions#2023-10-3114:22henrikBasically just flipping some CSS classes.
(cause
(reset! processing-atom true)
(transact-some-stuff-to-server …)
(reset! processing-atom false))
#2023-10-3114:22Dustin GetzDoes this idiom solve your problem https://electric.hyperfiddle.net/user.demo-chat!%43hat#2023-10-3114:23Dustin Getz(catch Pending e (dom/style {:background-color "yellow"}))#2023-10-3114:27henrikThe stuff I need to control sits in an awkward place, which is why I’m proxying status to an atom. Will true in the try and false in a finally do the trick?#2023-10-3114:29Dustin Getzi'm not sure that finally makes sense in electric#2023-10-3114:29Dustin Getzyou can e/on-unmount in the catch body#2023-10-3114:30henrikAh, that might be better then. I’ll give it a whirl.#2023-10-3114:35henrikThis does not work:
(try
(reset! processing-atom true)
(when-let [atom-ident (e/server (e/offload #(create-new-atom node current-space prototype account)))]
(router/route-to-entity! atom-ident))
(catch Pending _
(e/on-unmount #(reset! processing-atom false))))#2023-10-3114:35Dustin Getzfirst, put the reset! inside the pending body#2023-10-3114:35henrikAh#2023-10-3114:36Dustin Getzi dont know why it doesn't work though#2023-10-3114:36Dustin Getz(let [load-state (try (e/server ...) ::done (catch Pending _ ::pending))] ...) is maybe a useful idiom#2023-10-3114:37henrikMore context:
(dom/on "click"
(e/fn [_evt]
(try
(when-let [atom-ident (e/server (e/offload #(create-new-atom node current-space prototype account)))]
(router/route-to-entity! atom-ident))
(catch Pending _
(reset! processing-atom true)
(e/on-unmount #(reset! processing-atom false))))))#2023-10-3114:41henrikThis sets the atom to false initially, but does nothing else, it seems.
(e/fn [_evt]
(reset! processing-atom
(try (e/server (e/offload #(create-new-atom node current-space prototype account)))
false
(catch Pending _ true))))#2023-10-3114:45Dustin Getztry outside of dom/on (we don’t like dom/on it is dying after differential electric lands)#2023-10-3114:54henrikThis works:
(case (helpers/on! "click")
(reset! processing-atom
(try (e/server (e/offload #(create-new-atom node current-space prototype account)))
false
(catch Pending _ true))))
Where helpers/on! is:
(defmacro on!
([type]
`(nil->pending (dom/on! ~type identity)))
([type f]
`(nil->pending (dom/on! ~type ~f))))
It doesn’t work twice though, after it’s run the first time, it dies.
If I don’t init with pending, it’ll run on app boot.#2023-10-3114:56henrikI think maybe a couple of nested case is a more pragmatic approach at the moment.#2023-10-3115:30Dustin Getzwhat is "dies"#2023-10-3115:30Dustin Getzwhat is "nil->pending"#2023-10-3115:31henriknil->pending: (if (nil? value) (Failure. (Pending.)) value)#2023-10-3115:31Dustin Getzi dont understand why you would artifically throw pending here#2023-10-3115:31Dustin Getzand (Failure. (Pending.)) is i think not what you want it to be. You need to throw#2023-10-3115:32henrikRight, I’m reaching the limits of my understanding here. The goal is just to prevent triggering the else case of case on boot, after putting the work outside of on!.#2023-10-3115:33Dustin Getz(when click-event ...) ?#2023-10-3115:38henrikOh yeah, I’m over engineering#2023-10-3115:45henrikLike the one above, this works, but only once. The true branch of when won’t execute again for another truthy value, I presume.
(when (dom/on! "click" identity)
(reset! processing-atom
(try (e/server (e/offload #(create-new-atom node current-space prototype account)))
false
(catch Pending _ true))))#2023-10-3116:01xificurCdom/on manages the pending, if you catch it and don't rethrow it'll do weird stuff. I think you want
dom/on click
reset! atom true
e/server ...
e/on-unmount reset! atom false
#2023-10-3116:01xificurCNo try catch#2023-10-3116:05henrikWorks#2023-10-3116:08henrikNice, I’ll skip the case stuff then. I’m a little bit disappointed that I couldn’t get the “inverted” layout to work. But I’ve been there and failed previously as well.#2023-10-3118:30xificurCI think ui5 will solve that. Until then we have to stick with what works#2023-10-3116:28pezIs this just how the queue behaves? (I’m guessing I am tapping an empty queue?)#2023-10-3116:39Dustin Getzthere's a special calva command happening here right?#2023-10-3116:39Dustin Getzrun it line by line (tap becomes println), or run the whole test#2023-10-3116:40Dustin Getzi'm not sure what you're seeing, i will try to find out#2023-10-3116:41Dustin Getzrunning partial tests with the special calva command is going to leak processes, you must cancel each process#2023-10-3116:52pezYeah, if run them one by one or all of them, then it behaves as I expected.#2023-10-3117:06Daniel JompheDustin, this calva command to eval, once you get used to it, you start using it so often, it’s so productive. Worth getting used to it. Here are the variants.#2023-10-3118:12Dustin Getz@U0ETXRFEW i can't repro the 45350#2023-10-3118:12Dustin GetzI used eval from start of toplevel form to cursor#2023-10-3118:20pezI can’t reproduce it either. But I had been doing eval to cursor a lot of times before that, so maybe that is how to repro. (If the repro is any interesting, and simply not just a non supported use.)#2023-10-3118:21Dustin Getzi just wanted to improve the tutorial#2023-10-3119:11Vincentpez discovers electric, 楽しみ！ (Looking forward to these discoveries)#2023-10-3117:41Dustin GetzWe made a missionary quickstart, please try it and give feedback!#2023-10-3118:27Dustin GetzConnection details for the missionary talk starting shortly: https://www.youtube.com/watch?v=ZcsTWFTmUO0#2023-10-3119:50telekidsad I missed it, looking forward to watching the recording#2023-11-0114:59telekidThat link no longer works but this one does: https://www.youtube.com/watch?v=xtTCdT6e9-0#2023-10-3120:15VincentI added the ability for users to "upvote tags" that correspond to hyperlinks on http://NextApex.co but I still need to learn the tricks of the trade for catching the "upvote click" and not vacillating on it. There is some vacillation/wigglyness that occurs, I reckon due to the streaming nature of the differential. @dustingetz you mentioned there were some tricks, I will post the component code here although if you need more feel free to tell me#2023-10-3120:16Vincent(e/defn TagItem [id]
(e/server
(let [e (xt/entity db id)
xt-id (:xt/id e)
author (:tag/minted-by e)
tag-id (:tag/id e)
tag-minted-at (:tag/minted-at e)
title (:tag/title e)
target-id (:tag/target e)
upvotes-set (:tag/upvotes-set e)
upvotes (:tag/upvotes e)
downvotes-set (:tag/downvotes-set e)
downvotes (:tag/downvotes e)
u (if (not= "" online-user) online-user nil)
vw true]
(e/client
(dom/div (dom/props {:class "fr"})
(dom/div (dom/props {:class "fc"}))
(dom/div (dom/props {:class ["fc" "atag"]})
(dom/div (dom/props {:class "upvote-tag"})
(if (not (nil? u))
(ui/button
(e/fn []
(e/server
(e/offload
#(xt/submit-tx !xtdb
[[:xtdb.api/put
(if (and u (contains? upvotes-set u) vw) ;;vw is verified login
;; user has upvoted, remove their upvote
(-> e
(update :tag/upvotes-set disj u)
(update :tag/upvotes dec))
;; user hasn't upvoted, include their upvote
(-> e
(update :tag/upvotes-set conj u)
(update :tag/upvotes inc)))]]))))
(dom/props {:class (if (contains? upvotes-set u) "alreadyupvoted" "notyetupvoted")})
(dom/text "▲ " (count upvotes-set)))
(dom/div (dom/text "▲ " (count upvotes-set)))))
(dom/div
(dom/props {:class ""})
(dom/text title)))
(when (= "R" online-user)
(dom/div (dom/props {:class "fc"})
(ui/button
(e/fn []
(e/server
(let [target-entity (xt/entity db target-id)
updated-target (update target-entity :item/tags disj title)]
(e/discard
(xt/submit-tx !xtdb [[:xtdb.api/put updated-target]
[:xtdb.api/delete xt-id]])))))
(dom/props {:class "delete"})
(dom/text "✗")))))))))#2023-10-3120:21Vincent#?(:clj
(defn tags-for-newsitem [db newsitem-id]
(->> (xt/q db '{:find [(pull ?t [:xt/id :tag/minted-by :tag/id :tag/minted-at :tag/title :tag/upvotes-set :tag/upvotes])]
:where [[?t :tag/target nws-id]]
:in [nws-id]} newsitem-id)
(map first)
(sort-by :tag/upvotes >)
vec)))
The (sort-by :tag/upvotes >) sometimes makes the tags jump around to stay sorted. this makes the "vacillation" very apparent.#2023-11-0114:26WillHey guys, recently started working with Electric. Something that I have been thinking about is why Electric doesn’t throw errors or give warnings when using Clojure functions where it expects an Electric class? E.g. (dom/on "change" (fn ... instead of e/fn or an e/defn Is there a reason for this?#2023-11-0121:12Dustin Getzbecause it’s not trivial to determine that in the current impl (and it’s not like clojure makes this kind of thing easy)#2023-11-0121:13Dustin Getze/fn compiles to missionary flow which is encoded as a thunk, so both are fns actually at runtime #2023-11-0212:09WillI see, thanks#2023-11-0308:54avocadeAt least there's a nice warning if trying to use eg a (e/server) call inside a normal fn 👍
Continuing to add more errors/asserts overall, preferably as close to the offending site as possible, would be much appreciated of course (top on the Clojure wishlist still I think as well: "please improve the errors/exceptions"). Especially when it's this early stage, and both docs and best practices are still very sparse. Guide rails 🚀#2023-11-0212:22henrikI’ve started getting these guys: Connection to client lost after 90s. Closing socket (http-kit adapter). Any idea where I should start hunting for the source problem?#2023-11-0212:42Dustin Getztry upgrading to master, we will log a ticket to investigate#2023-11-0212:42Dustin Getzis this in local dev or prod only?#2023-11-0212:42xificurChttps://github.com/hyperfiddle/electric/commit/5a3799bdca4f70aed0dc51ac98ff9728dbffacae this is on master only, you should try it out to see if it resolves your issue#2023-11-0212:45henrikI’ll confirm, but I think it’s local only.
Thanks both, I’ll give it a go.#2023-11-0214:48henrikCan confirm I see it deployed as well. But I’m not seeing it after using master.#2023-11-0214:49Dustin Getzok keep us updated#2023-11-0307:36henrikI left the computer running during the night to see what happened. It looks like it had some hiccups during the night#2023-11-0307:37henrikOn the other hand, I’m not seeing the same thing with the deployed version.#2023-11-0308:57avocadeCould have something to do with new behavior in macOS Sonoma maybe? Aggressive cleanup of bg resources that the OS thinks is burning energy without need, etc. Apple has been aggressive in minimising the energy usage in the latest iterations (which is mostly nice of course).#2023-11-0312:28Dustin Getzthis 5 minute interval is symptomatic of Chrome aggressively throttling timers (of background tabs IIRC)#2023-11-0312:29Dustin Getzthe heartbeat is of course controlled by a timer#2023-11-0312:30Dustin Getzyou might verify that your http kit adapter (presumably you customized) is the same as the one in master#2023-11-0312:31henrikAlright, that makes sense then. No customization, straight up using the one provided by Electric.#2023-11-0215:54Hüseyin YavaşHi, I have a question about routing. I started to use react components and I tried to call onClicked function in the component but I got a protocol error. I isolated the error in a project. In the project, app02 function should be called when the user click the image. The project link is https://github.com/mhyavas/Student-Registration-Electric/blob/main/error-debug/src/app/t01.cljc. The error is in the image. Can you help me about it?#2023-11-0216:03xificurCon line 35 you're trying to call an electric function (history/navigate!) from a clojure function. It's like trying to do e.g. 1 + (inc 2) in js#2023-11-0216:07xificurCon further inspection it's the other way around, history/navigate! is a clojure function but you're invoking it as if it was an electric function, with a trailing dot#2023-11-0309:12Hüseyin YavaşI was trying different methods to figure out what's wrong. I remove the dot and doesn't change the error. Also, I tried to call it in electric function but it gave another error.#2023-11-0309:29xificurCIs it required you navigate from within reagent? The IMG could be an electric component and you'd save yourself a lot of headache#2023-11-0309:31xificurChistory/!history is an electric dynamic var, which will be nil/unbound in clj(s) in this case. The cheapest trick would be to pass it as another argument to the with-reagent call and then call (history/navigate! router ...)#2023-11-0310:51CanIs it possible to create a state to handle events for react comp? When clicked or the event is thrown then watch the state in Electric going to be changed and you can call your “history/navigate! call” with it. @U09FL65DK I think yes, it is possible but not sure that is the best practice or not.#2023-11-0311:07xificurCsure, e.g. define an atom, reset! it in cljs code and e/watch it in electric#2023-11-0320:15Hüseyin YavaşThank you guys, I'll try to define atom#2023-11-0314:43henrikI’m working on our AI stuff now. I’ve wrapped our old AI infra, which returns a stream of messages in the form of maps, in an m/observe.
I’m relieving this with a function that merges into a single message, if necessary.
For the semantics of Electric, is this enough to ensure that I don’t lose messages? For example, let’s say I take those results (on the server) and merge into an atom (client side), will the atom eventually contain the complete state?
The other alternative (that I can see) being that I could accumulate in an atom on the server, which I read from the client.#2023-11-0316:09xificurCGenerally electric runs in continuous time, i.e. you cannot rely on seeing all messages. If you show some code we can discuss more concretely.
I think we will have a better answer in the next version of electric, which will operate on diffs and allow weaving in streams of data. To be seen.
Accumulating on the server into an atom or db, in missionary discrete flows, and querying that from the client is guaranteed to work.#2023-11-0316:20henrikSomething like this#2023-11-0316:30xificurCThe latter is safer.
I'd also consider moving the swap! inside the observe code, if it's running for side effects only it doesn't need to return values back to electric.
(new (m/observe (fn [!] (! nil) (swap! some-atom merge ..) #(cleanup))))
This will return nil on mount and never update, which is OK for side effects.#2023-11-0316:34henrikThanks @U09FL65DK#2023-11-0421:47nivekuilwhat might an electric-dom3 look like? i'm designing something similar, would be interested to hear any lessons from dom2#2023-11-0421:53Dustin Getzthe problem with dom2 is the event handing is imperative, adding latency to the reactjs change callback pattern has really confusing semantics #2023-11-0421:55Dustin Getzit’s also plausible that the event side of views should be done in discrete time, which means no network transfer in response to events except at the root of the component tree#2023-11-0421:56Dustin Getzthat’s my 2c what is yours?#2023-11-0421:58Dustin Getzalso optimistic updates infrastructure is needed#2023-11-0421:59nivekuilaside from what's blocked by differential electric, are you satisfied with (or at least have no current plans to change) the api?#2023-11-0422:09Dustin Getzum a better props syntax would be nice but the requirements are stringent, it cannot damage expressive power#2023-11-0422:37henrikUIx etc. have special considerations in their macros for props delivered as a map literal in the first parameter position of DOM constructors.
But it’s also quite powerful to be able to return props and treat it as a graph node in some cases.
A #props reader macro? #2023-11-0715:29nivekuil it’s also plausible that the event side of views should be done in discrete time, which means no network transfer in response to events except at the root of the component treeI think I agree with this; I made callbacks defer their effects by putting them into a global event queue that gets flushed every frame, inspired by https://bevy-cheatbook.github.io/programming/events.html I think it's pretty clean, seems to have fixed the glitches i was seeing by making all the reactive effects happen at known points and also paves the way for split client/server data#2023-11-0715:47henrikInteresting @U797MAJ8M. What glitches were fixed by using an event queue?#2023-11-0715:52nivekuilchanges being dropped etc. probably the same sorts of glitches dustin is referring to with react-style callbacks that they want to get rid of with differential electric, and making multiple watches on an atom; you can only have one reactive system that fully controls when all things happen, you can't mix dataflow/execution semantics or things become impossible to reason about#2023-11-0715:54nivekuilthe problem is more visible in ecs (because it forces you to think explicitly about data dependencies and execution order, i'm bridging odoyle and electric for dynamic reactive state) than frp but should exist in general#2023-11-0715:58Dustin Getzidk what glitches you are talking about, the ui4 library has no known bugs or glitches#2023-11-0716:03nivekuilwell yeah, ui4 bridges callbacks to electric/missionary land#2023-11-0500:38grounded_sageIs there any solutions for handling a lot of atoms being watched?
I'm wondering if something like this would be good.
{:client {mouse-pos !mouse-pos
on? !on?}}
Thoughts?#2023-11-0616:15xificurCWhat is your use case? And what is the problem with a lot of atoms being watched that requires a solution?#2023-11-0616:41grounded_sageNo use case. Just reduction of text in file.
Just sugar 😆 #2023-11-0500:40grounded_sageI guess this could be confusing because it would be (e/client (e/watch !mouse-pos)) and having client in front could cause confusion as to whether mouse-pos is available in e/server#2023-11-0616:10Vincent@dustingetz a little off topic but what do you use for screen recording? I am thinking to make some "walk through" videos of my electric code and currently i'm doing (2x quicktime)#2023-11-0617:07Adrian SmithI used Wondershare demo creator to create a RCF video in 2021
seemed ok to use at the time, I'm not a power video editor though#2023-11-0617:36Vincentthanks Adrian 😄 will check it out now#2023-11-0620:08telekidhttps://www.loom.com/ is worth a look#2023-11-0621:25grounded_sageVimeo has a pretty good free recorder these days as well#2023-11-0617:35henrikI need a better way to send an access-token along when establishing the Electric websocket connection. What’s the suggested way to do this?#2023-11-0617:37Vincentcurrently represented as session id in my knowledge#2023-11-0617:37Vincentin the examples^#2023-11-0617:38Vincentideally the access token is stored... clientside in browser storage, in header of every request? both?#2023-11-0617:39Vincenttangentially related: i'm using tuple <timestamp, username> to derive a <userhash> and pass all three when the user does stuff. Probably a better way exists.#2023-11-0617:48henrikI was pretty sure that you can’t use headers with a websocket connection, and have to rely on a query-string or some other mechanism. How do you supply these headers?#2023-11-0617:53henrikI believe the session ID used in the examples is just the standard session ID attached to all websocket connections.#2023-11-0617:56Geoffrey GaillardAs far as I know, it is not possible to set headers on the WS UPGRADE request.
Is it ok to pass your token as a query param?
If no, then you need a two-step mechanism:
1. Issue a request to /set-token,
a. pass your token as a header
b. generate a unique id (e.g. uuid),
c. store the pair (uid, token) into some hash-map
i. ensures the pair is short-lived - clean it up after a few seconds (e.g. with a missionary task or make your map a TTL cache)
d. return the uid
2. pass the UID as a query param in the WS connection url
a. get the corresponding token from the map#2023-11-0617:59henrikIn this case, a third party (Auth0) handles all the storage etc., so I literally only have to find a non-insane way to have the access-token travel along with the WS connection request.#2023-11-0617:59henrikBut 2 assumes that I can manipulate *ws-server-url* in Electric. Is this supported?#2023-11-0618:08Geoffrey GaillardNot in Electric, but you can rebind *ws-server-url* before calling hyperfiddle.electric/boot , so before the program starts.#2023-11-0618:22Vincentoh that looks like a very elegant solution (!)#2023-11-0618:22Vincentmaybe you can write a blog post about how you achieve it#2023-11-0618:50JWhat about the cookie?#2023-11-0619:26henrikNot the kind of thing you want to leave hanging around in a cookie. It’ll go stale if your cookie lives longer than the access token it holds.#2023-11-0618:34Adrian SmithOpenAI are letting people create custom gpts, I remember Dustin once said it would be good to have a bot that could help with support check it on their devday video#2023-11-0700:26Jordan CalderwoodYeah a dialed in Electric GPT would be amazing for helping with the minutia #2023-11-0721:21joshchohttps://clojurians.slack.com/archives/C7Q9GSHFV/p1699392001098329#2023-11-0622:24weianybody have a working electric+rama clojure api example of subscribing to changes on a pstate? having some trouble converting the callback of foreign-proxy to something electric can use (missionary observable?)#2023-11-0708:06JI’m doing this:
(defn make-reactive-query [path pstate]
(->> (m/observe
(fn [!]
;; Init
(! nil)
(let [cp (r/foreign-proxy-async path pstate {:callback-fn (proxy-callback !)})]
#(.close @cp))))
(m/relieve {})))
There is also a thread about this: https://clojurians.slack.com/archives/CL85MBPEF/p1698062851851949#2023-11-0718:15weithank you. and what is proxy-callback in your :callback-fn?#2023-11-0718:21wei(my guess is, it's not needed if i'm observing the whole collection and not looking at diffs?)#2023-11-0721:20joshchoI am going to an emergency post-devday hackathon. Looking for some teammates to build with electric.#2023-11-0721:20joshchohttps://partiful.com/e/ccbe5YFUSytVsqjUAH6s#2023-11-0721:20joshchoI can’t be there in person (flying out today) but would love to build cool things in electric together!#2023-11-0800:26grounded_sageSeems that using tmdjs causes missionary to cancel. All I do is require it in and it causes electric to completely crash in the browser.
Repro - https://github.com/groundedsage/electric-tmdjs#2023-11-0800:28grounded_sageLooking to use techml for sending large timeseries datasets to the frontend plus performing a bunch of compute required for rendering into datavis.#2023-11-0813:45grounded_sageI would like to have infinite scroll and panning in my datavis.
I am currently thinking I need to structure things this way
• all data (backend)
• buffered data window in frontend
• virtual (visualised/rendered) window#2023-11-0813:48grounded_sageUse Rama for all data. For the data I want in the frontend put that into a techmldataset and then transmit it to the frontend using the built in transit. Then read that into a tmdjs dataset in the frontend.
Following that I would do a filtering for the virtual window I am rendering and then also run scaling on the values. Also calculating moving averages.
Then put each entity into a sequence of maps for Electric rendering inside dom/svg#2023-11-0814:03Dustin GetzAs I said in DM: For our paid design partners we are developing efficient virtual scroll patterns that are designed to scale to large datasets. But unfortunately capabilities like this we can only offer to our commercial partners at this time.#2023-11-0814:08grounded_sageI sent all these messages around the same time. I respect the work required here and understand. It’s hard work.
I’m happy to keep noodling on this problem and find a solution. The only blocker there is tmdjs causing missionary to cancel when loaded into the namespace. But I can also work around that for now. #2023-11-0814:15grounded_sageI can also try digging into what is causing the tmdjs/missionary issue. But unsure where to even start looking for that. #2023-11-0817:33JHi guys! I have something like this:
(e/defn Board [route-name route-match]
(case route-name
:foo (Foo. (:id route-match)
:baz (Baz. (:id route-match)))
Let’s say route-name and route-match change now. In this case, even if the route-name is :foo the branch :baz of the case will be fire, right?#2023-11-0818:00xificurConly the active branch fires, just like in clojure#2023-11-0820:51JIt’s weird because here:
(e/defn Board [route-name route-match]
(e/server
(let [raw-user-id (get-in e/*http-request* [:cookies "cookie-id" :value])
user (retrieve-logged-user raw-user-id)]
(e/client
(binding [current-user user]
(dom/div
(Header.)
(dom/div
(case route-name
:foo (quiz-list/QuizList.)
:baz (quiz-item/QuizItem. (-> route-match :path-params :id))
:bar (do
(prn "THIS IS PRINT WHEN route-name IS :foo")
(quiz-session/QuizSession. (-> route-match :path-params :id)
raw-user-id))))))))))
the :bar branch is fire even if the route-name is :foo. I don’t understand the cause of this.#2023-11-0820:56JTo be clear the branch :foo is displayed but the prn into the branch :bar is also displayed.#2023-11-0908:12xificurCI cannot reproduce that, the below code only taps 1 and doesn't print#2023-11-0908:13xificurCmaybe route-name starts as :bar first? Try printing it#2023-11-0908:34JThanks! I continue to investigate.#2023-11-0819:19CanHello! I wondering is it possible to disappear that vector from screen ? I using history/router in my project and I seeing the params on screen and of course local host link. So if possible I dont want to see that. 🙂#2023-11-0819:39xificurCdidn't you copy it from one of the main entrypoints?
(dom/pre (dom/text (contrib.str/pprint-str history/route)))
#2023-11-0820:25CanYes I deleted that but still link is same. Isn't that possible to hide arguments?#2023-11-0820:26CanAnd thank you for the help as always. 🙂#2023-11-0820:40xificurCGlad to help! The link is the storage in this case, e.g. to persist over tab refresh#2023-11-0820:57CanI am scared that if a user changes the link's param part with some mock data is it going to make trouble for the app? Because I can change users from the link 😄#2023-11-0820:57Cansome kind of baby-level security problem. If I am not wrong.#2023-11-0907:48xificurCelectric doesn't change the security considerations of one's application. The backend should still validate all user input, including the URL#2023-11-0907:49xificurCand with publicly available data it's not a problem, e.g. you can change the username in a github link, reddit etc. to get to one's profile page#2023-11-0920:10CanI see, I already prevented and restricted improtant datas in backend but I just wondered is it possible to they params on link or not. 🙂 Thanks for the help! 🙂#2023-11-0821:37CanI have one more question: is there any supported event list exist in electric? I used before keydown - keyup - click -change but now I need to use load events and some mouse events.
like: Load Events
• DOMContentLoaded: Fired when the initial HTML document has been completely loaded and parsed without waiting for external resources like images.
• load: Fired when the entire page, including all images and external resources, has finished loading.
(dom/on "DOMContentLoaded" (fn [] (swap! !state assoc :category (keyword "service" (str (.-value dom/node))))))
(dom/on "load" (fn [] (swap! !state assoc :category (keyword "service" (str (.-value dom/node))))))
(dom/on "change" (fn [] (swap! !state assoc :category (keyword "service" (str (.-value dom/node))))))
#2023-11-0821:38Dustin Getzuse dom/on! with clojure.core/fn, dom/on is for e/fn#2023-11-0823:51Dustin Getzall events should be supported, if something is wrong the dom/on! source code is straightforward to understand and can be adapted to some other dom API#2023-11-0823:46Tommy JollyboatHello, I have architecture-related questions. In the future, I'd like my app (a multiplayer party game) to be able to run in "Hybrid Mode" on both LAN and Cloud simultaneously, sharing state between 2 server instances, so clients can interact transparently as if there were one server with 2 URLs. There's no rush for this feature, I'm happy to wait a year or two. (For context, it'll run for only 2 hours at a time, with 10-2,000 clients, and fairly low throughput).
Question 1: When the Electric Examples App runs on multiple http://fly.io "machines", can those instances "see" each other, or is my instance of the multiplayer chat only sharing state with other clients in my region?
Question 2: Would I achieve my Hybrid Mode by running 2+ servers (when Elec implements that), or by having one of them essentially be an event-forwarder, that round-trips everything to the "real" server?
Question 3: I'm about to replace my hand-rolled state-atom with DataScript or Datomic Local (or Datomic Pro if needs be). If my Hybrid Mode is a 2-server solution, does it require Datomic Pro running decoupled on the cloud somewhere, or will Electric be able to share & update the other in-memory databases between instances?
In a way, these are all the same question, but my understanding of both Electric & distributed systems is severely lacking. TIA!#2023-11-0823:53Dustin GetzQ1: fly instances don’t share state, yes they can reach each other but shared global state is your problem, likely you want to have each fly region connect to the local AWS region and use a service that advertises cross region writes#2023-11-0823:54Dustin Getzbasically Electric is your web tier only, in the end Electric is just a protocol there is no magic or new physics#2023-11-0823:59Dustin GetzQ3: Datomic is a single region datastore, and Electric servers do not talk to each other (theoretically we could assist with this but still it will not get you the efficient globally distributed multi master writes that you are looking for)#2023-11-0900:00Dustin GetzGoogle Spanner is a good starting point for multi master writes, (note this is not my domain of expertise)#2023-11-0900:02Dustin Getzor if this is overkill maybe a CRDT? idk#2023-11-0900:04Tommy JollyboatVery comprehensive, thank you 🙂 I think CRDT or having a single master will be fine, I just want to avoid making mistakes early on that will require huge rewrites later.#2023-11-0900:27nivekuilI recommend that you use a dedicated central store to handle shared state instead of rolling your own, but you could try hazelcast if you don't want a third party server. depending on your requirements the latency of something like redis/scylladb is quite low already#2023-11-0902:07Dustin Getzi also like hazelcast for this now that i understand better the problem#2023-11-0909:19Mario Trost> Electric is just (...) magic or new physics
I knew it!#2023-11-0911:12andersmurphy> For context, it’ll run for only 2 hours at a time, with 10-2,000 clients, and fairly low throughput
With those numbers I wonder if it will be much simpler to run a single server. You can use something like https://github.com/juji-io/datalevin for an in process datomic/datascript like, which would simplify your ops considerably. Datalevin meshes really well with electric in my limited experience. It also has a server mode if you need to scale up to multiple instances, so it won’t lock you in to a single server if for whatever reason your project is wildly successful.#2023-11-0919:08Tommy Jollyboat@U0JEEGD4N Yes I think that's the answer - I've made a note of Hazelcast for later, but for now I'll stop overthinking and stick to the simplest thing that works 🙂 Datalevin looks good, I'd forgotten that one!
I noticed you're in Edinburgh BTW - my goal is to do a run at Edinburgh Fringe with whatever interactive thing I end up making, so if you're there in August, it'll be Liquid Rooms on Victoria St 😉#2023-11-1010:51andersmurphyOh, awesome. Clojure is making the jump to the fringe! - I do think electric is a natural fit for digital party games so i’m excited to hear/see how it goes.#2023-11-0907:45weianyone have tips on organizing complex uis so they compose well? also, higher abstractions for building forms with validation? (currently using atoms for individual fields)
i suspect HFQL will help with the second one, but we'll have to wait for that i guess#2023-11-0908:20xificurCare you struggling to compose UI components? Electric's main primitive is a lambda (e/fn), which should compose out of the box.
re forms with validations - we don't have demos to show here yet as most of these were client specific. Using an atom per field works for field-level validation, you could also use an atom per form for form-level validation#2023-11-0909:25weiyes I'm really enjoying being able to compose UI with functions. I'm trying to figure out routing at the moment which I've heard folks use reitit for. If anyone has a minimal routing example I'd like to see it#2023-11-0909:27weiAlso for managing app data is it best practice to use atoms scattered around? Or a client-side app db? Or is it recommended to use a streaming database and be reactive on the server-side db?#2023-11-0909:28xificurCif you search in:#hyperfiddle reitit there's a setup from @U050CJFRU and @U06B8J0AJ#2023-11-0909:32xificurCthere is no best practice, all of it works. Depends on your requirements really. Do you need data to stay on the client? Do you need it to survive machine restarts? Should the data be visible across multiple tabs? Multiple devices? Multiple users?#2023-11-0909:34weiI suspected that might be your answer 😅 guess I'll need to experiment. Thanks for the help!#2023-11-0909:34xificurChaha sorry!#2023-11-0909:34weiNeed persistence so will need a server side db#2023-11-0909:35weiI'm excited to try out Rama, but planning to use it for a hobby project first#2023-11-0909:36xificurCyeah, electric should work great with rama. Once differential electric lands it will fit like a glove#2023-11-0913:34telekidIt would be nice to have editor plugins that highlighted server->client transfers to help prevent accidental secret leaks:
(e/defn Example
[]
(e/server
(let [secret hunter2]
(e/client secret ;; 'secret' would be highlighted
))))
I could even imagine a linter rule that required one to highlight transferred values somehow (e.g. a reader tag, a casing convention, etc.)#2023-11-0913:46xificurCgiven enough resources (money, time) I'd build an electric IDE with many more features... Well, pulling my head back from the clouds, maybe one could extend clj-kondo or tree-sitter to get some functionality. I also wrote a https://gitlab.com/xificurC/hf-electric.el/-/blob/master/hf-electric.el?ref_type=heads for emacs to add a colored background to client/server code.
To programmatically prevent leaks one can hold secrets in a non-transferable value, something like
(ns secret)
(deftype Secret [v])
(defn seal [v] (Secret. v))
(defn open [s] (.-v ^Secret s))
and use as
(e/server
(let [secret (secret/seal "hunter2")]
(e/client secret))) ; <- one will see a transfer warning and receive nil#2023-11-0914:36telekidah, very clever!#2023-11-0914:40telekidyeah and now that I think about it a bit more, it isn't clear statically (without compilation, at least) where transfers happen#2023-11-0915:09xificurCour analyzer infers transfer points without compilation. In theory the analyzer could in the future provide the analysis information through some API. Not on the roadmap#2023-11-0919:49Dustin Getzif this is important to you, wrap secrets in a deftype and do not install a serializer for the deftype#2023-11-0919:50Dustin Getzor write secret processing code in a .clj file#2023-11-0914:41henrikWhat’s the difference between e/inhibit and e/discard?#2023-11-0915:18xificurCyou can use e/discard for now, the difference is only interesting outside of electric. At the missionary layer e/discard is a flow of nils whereas e/inhibit is a flow that only emits a single nil#2023-11-0915:25henrikAh, interesting, thanks for the explanation.#2023-11-0915:27Dustin Getze/inhibit is not useful in electric v2#2023-11-0915:27henrikGotcha. Is it actively harmful?#2023-11-0915:28Dustin Getzor v3 either, it's a temporary utility for a client project that pulls forward some differential stuff into electric v2#2023-11-1008:31avocadeNice, this caught my eye just yesterday as well. Have to remind myself that alpha means API will change a lot. #2023-11-1007:26henrikHere’s the latest speed bump in my adventures of wrangling Auth0 under control: I need to set *ws-server-url* dynamically when reconnecting. If the access token has expired before the next time that Electric reconnects, I effectively need to provide a new URL. I’m currently thinking that I need to copy all code relating to reconnects into our repo, or is there another way to do this? Bonus: the Auth0 access token API returns a promise—it’s not synchronous (fixable with m/observe maybe)#2023-11-1007:46xificurChttps://github.com/leonoel/missionary/wiki/Task-interop#futures-promises-1 to wrap a promise into a task
The rest I'm missing context to answer. Why do you need to provide a new URL, where is the auth0 access token stored, what happens if electric reconnects and the token expired, how/when do you initialize the auth0 access token#2023-11-1007:50henrikThe token is stored in indexeddb, and Auth0 appears to have some kind of background task to refresh it just before it expires. There’s a function to retrieve it when needed (everything that manipulates indexeddb returns a promise, hence so does the Auth0 function).
The access token is initially set by calling a login function which redirects to a login page if required etc.
The URL is set to something-something?access-token=…&HYPERFIDDLE_ELECTRIC_CLIENT_VERSION=… in order to deliver the AT synchronously with the request.#2023-11-1007:53henrikIf Electric attempts to reconnect with an expired token, authn/authz middleware rejects the request.#2023-11-1007:55henrikHence it’s possible to get around the problem by reloading the page, since that’ll set a URL with a new access token. The problem appears specifically in the situation where a) AT is expired, and b) Electric needs to reconnect for whatever reason.#2023-11-1007:59henrikMy inclination is to fix this the same way we did in our vanilla websockets: retrieve access token (and CSRF token, which also might have changed) -> construct URL -> attempt to connect. I’m open to suggestions, however.#2023-11-1008:13xificurC> Auth0 appears to have some kind of background task to refresh it just before it expires
how does it expire then?#2023-11-1008:14xificurCif you see a clear path forward that meets your requirements I suggest you try it out#2023-11-1008:21henrikI’ve bound *ws-server-url* to the URL something-something?access-token=a1&HYPERFIDDLE_ELECTRIC_CLIENT_VERSION= at app boot. In the meantime, the access token changes to b2, let’s say. Just because it’s b2 in indexeddb doesn’t mean that *ws-server-url* is automatically aware of this, unless I do something.
Electric will still try to reconnect on the a1 token, unless I can retrieve the new token inside of the reconnection cycle, and construct a new URL.#2023-11-1008:23henrikI wouldn’t say it’s a good path, since it means copying whatever is necessary from the Electric repo over to our codebase, but clear—sure.#2023-11-1008:34xificurCI see. So you're already constructing your own URL and then what, you use binding or set!?#2023-11-1008:35henrikbinding#2023-11-1008:35henrikI’ve copied over do-browser for this purpose.#2023-11-1008:36xificurCCan you watch / get notified when the token changes?#2023-11-1008:37henrikI’m not sure, but let’s say I can. We can poll it, if everything else fails.#2023-11-1008:39avocadeCrazy idea: fork the auth0 SPA lib and clojurify it so we can have proper atoms containing the tokens (including refresh token), and hook it in to their existing event/refresh-timer system. Then we could watch the atoms. Haven’t looked at the lib for years though.
@U06B8J0AJ polling is probably a good fallback anyhow, so maybe start with that? Just to get it working, and then make it great “later”. Sad but pragmatic. #2023-11-1008:41henrikNot sure we have to: I’m pretty sure we can listen to changes in indexeddb. Ultimately, it would be pefectly fine IMO to just run (.getTokenSilently client) on each reconnection attempt (which will return a new or the current token as necessary, as a promise).#2023-11-1008:43xificurCI was thinking of reading the value and set!ting it into *ws-server-url*, so you don't have to fork anything. But I see boot-with-retry closes over the value, so that probably wouldn't work#2023-11-1008:45henrikRight, I think that would be open to some race conditions in some rare circumstances, but would probably work most of the time.#2023-11-1008:45henrikModulo the closure of course.#2023-11-1008:47xificurCI'd say just fork the file or redefine the function and get it to work first#2023-11-1008:47henrikGreat, I will do that. I just wanted to check that I wasn’t immediately picking the path with the most friction.#2023-11-1008:58avocade👍#2023-11-1008:59avocadeThanks Peter for your quick input as always 🙏#2023-11-1009:28Geoffrey GaillardToday, a running electric program will always reconnect to the *ws-server-url* it first connected to.
Alternatives to you issue are:
1. Manage Auth token lifecycle outside of electric.
2. On token change, call a home-made variant of https://github.com/hyperfiddle/electric-starter-app/blob/main/src/user.cljs that would :
◦ stops the reactor
◦ rebind *ws-server-url*
◦ restart the reactor
3. fork and adapt electric_client.cljs to add an indirection.
Like you and Peter already talked about, I think 3. is the right thing to do#2023-11-1009:36henrikThanks @U2DART3HA. I’ve thought briefly of 1, and I have a feeling I would be patching race conditions for the foreseeable future, not to mention the complexity involved in adapting the stack of ring middleware to somehow jive with a token coming in via a side channel and so on.
2 would, I think, introduce Electric restarts where none are needed. So 3 seems like the way to go for now.#2023-11-1015:13telekidThis is probably not all that relevant to your situation, but I'm noting anyway since it might spark a new idea. I've written my own code to https://gist.github.com/telekid/3eff1030756f67237ae1932540f44ef5. My app supports the oauth "authorization code" flow which means I end up storing tokens on the server rather than in the browser, which might not be what you want. But it works well and means that all of my "logged out" logic along with the login flow can also be in electric – being logged in or logged out is just another bit of state within my app.#2023-11-1015:15telekid(I'm by no means an outh expert, please lmk if you see problems with this strategy)#2023-11-1015:16henrikYeah, if you don’t rely on authentication anywhere in the server router middleware, doing it entirely in Electric would be fine.#2023-11-1015:18henrikUnfortunately, by the time we hit Electric it’s “too late” for us.#2023-11-1215:46henrikFWIW, this is what I ended up with.#2023-11-1104:24VincentI want to define a :version/current for the app (current version) and have an (e/watch on the db ref and have it auto reload the page when the version shifts, i reckon this is doable with e/watch on a db call but i'm having trouble getting it to work#2023-11-1104:45Dustin Getzelectric bundles functionality to refresh on new build deployed, are you looking for something different than that?#2023-11-1104:47Vincenti want to trigger a javascript location.reload when i update a version number manually#2023-11-1104:58Dustin Getzto be clear, it already reloads in prod (not in dev) when the source code changes, will the database version ever change when source code didn’t?#2023-11-1105:02Vincentoh, it does? how can i tell it to be prod and not dev mode?#2023-11-1105:05Vincentand, to answer your question, the version will only change when code changes (-- the option to force reloads only necessary if auto-reload not a thing, since some clients hold onto stale .js asset)#2023-11-1112:58Dustin Getzsee https://github.com/hyperfiddle/electric-starter-app#2023-11-1116:11VincentI want hot code reloading and force my users to get latest version 😅#2023-11-1116:14Dustin Getzyou want hot code reloading in prod?#2023-11-1203:14VincentDanger is my middle name.#2023-11-1213:08Dustin Getzi think dev builds are already reloading changed JS for all connected clients every time something changes. However expect many glitches related to caching. Perhaps you are experiencing one#2023-11-1209:47Piotr RoterskiHello fellow electricians! 👋 ⚡
I've run into an issue with hyperfiddle.electric-ui4/button firing its handler function more than once when the function updates the state that function handler depends on.
In the simplest reproducible example, a button handler updates a server-side atom (the issue does not happen when atom is client-side only), but since handler depends on the atom's value, the handler gets executed twice (or more in more complex scenarios): once with initial state and then with its updated state.
Please take a look at the gif below and here's https://github.com/hyperfiddle/electric-starter-app/compare/main...roterski:electric-starter-app:2023-11-12-button-firing-twice-bug-showcase.
Is this a known issue? I could probably circumvent it somehow, but I'd argue that's an unexpected (and undesired?) behavior of a ui/button that can lead to pretty painful bugs (e.g. double-firing backend transactions).#2023-11-1212:47Dustin Getzknown issue, search for many discussions explaining what’s happening, use e/snapshot to work around. next version of electric-ui is callback free #2023-11-1212:59Piotr Roterskithanks! I've managed to work around it with e/snapshot like this:
(e/defn Page
[]
(e/server
(let [!state (atom 0)
state (e/watch !state)]
(e/client
(ui/button
(e/fn []
(let [state (e/snapshot state)]
(println "client CLICK " state)
(e/server
(println "server CLICK " state)
(swap! !state inc))))
(dom/text "button"))))))#2023-11-1213:00Dustin Getz note that the transaction here (swap!) does not depend on state, the thing causing the cycles are the printlns. if you remove the printlns you don’t need snapshot#2023-11-1213:01Dustin Getzyou need to understand this to use ui4 unfortunately, ui4 is robust to latency and concurrency but only if you understand the rules#2023-11-1213:01Dustin Getzui5 will be easier#2023-11-1211:28siddharth yadavI am making a svg canvas with nodes and edges connecting these nodes. Each node has a location on the canvas, directed edges are defined in terms of "to" and "from" ids of nodes. I am not sure what is the electric way of updating the position of edges in the canvas is.
When some node is moved in the canvas I update its position in the server and watch for the changes in the client. Should I do the same for edges or should I use m/observe and listen only on the client side?
I think I can model the data in another format so instead of directed edges are defined in terms of "to" and "from" ids of nodes I will store the incoming and outgoing edges in nodes itself.
So yeah I am confused, on one hand I have all the freedom to model as I want because of electric but on the other hand I am thinking is this the electric way, should I change my thinking to better use the power of electric.#2023-11-1211:32siddharth yadavAs of now this is all in prototype stage, I am building for future where I would have millions of these nodes and even if I would be seeing some very small subset of them they could change because another agent changed some other part of the graph and maybe that changes some part of my view#2023-11-1213:05Dustin Getzplease post code (i mostly read this on mobile so please post in a mobile friendly way, use the file attachment feature for longer snippets)#2023-11-1213:19siddharth yadavHere I attached the whole code part of it is some logic to convert b/w different coordinate systems and calculating position on canvas when circle is moved. view, circle and line function are the ones that would be directly related to the problem#2023-11-1806:48siddharth yadavI was not using it previously, but since you mentioned I re-read my code, docs and some chat. I was using e/for-by under client but the values are on server so I fixed it and now using value from the server like so (14 loc)#2023-11-1806:51siddharth yadavNow the UI is lagging more I think due to the nested for-by.#2023-11-1213:33grounded_sageearly stage but I'm learning datavis and creating a datavis gallery in electric
https://github.com/groundedsage/electric-datavis-gallery#2023-11-1213:35grounded_sageI'll be using this as a way of exploring the idiomatic electric way to do data visualisations.#2023-11-1213:38grounded_sageI wasn't able to figure out how to determine when an element is mounted to the dom and opted for a timeout to start the animation. How do I change this code so that it runs when mounted.
https://github.com/groundedsage/electric-datavis-gallery/blob/main/src/app/animation/connected_scatterplot.cljc#L144-L151#2023-11-1219:03Dustin Getzyou can capture a ref to dom/node like this:
(let [el (dom/svg ... dom/node)] ...) the key is that the dom syntax macros return the final child#2023-11-1219:35Dustin Getzyou can also move the .beginElement effects into the body of svg/animate where dom/node is available, giving you a dependency on it#2023-11-1219:35Dustin Getzin your case i think that is sufficient?#2023-11-1219:36Dustin Getzfor harder use cases if establishing a causal dep in the DAG is too difficult you can use the MutationObserver API, at least until we have time to take a look at making this better#2023-11-1317:23grounded_sageCool. The returning dom/node into a let binding is a good trick. It’s needed to actually get the length of a line because calling it inside that is not available.
I still haven’t quite landed on animating after everything is loaded. By the time the visualisation renders its like 1/3 of the way through the animation.
I’ll keep playing with it 🙂 #2023-11-1317:49grounded_sageLooking more at mutation observer I might need that. #2023-11-1614:06grounded_sageSolved this using Web Animations API and capturing dom/nodes#2023-11-1218:18weihow do you guys deal with uuid format differences between clj and cljs?
;; throws java.lang.IllegalArgumentException: Invalid UUID string: 100
(e/client
(ui/button (e/fn []
(let [id (uuid "100")]
(e/server id)))
(text "press me")))#2023-11-1218:20grounded_sageI use nano-id#2023-11-1219:01Dustin Getzcore has (random-uuid)#2023-11-1220:52joshchoHow much will end-user “api” change with differential electric?#2023-11-1301:39Dustin Getzhopefully none or minor changes to current APIs, maybe some additions#2023-11-1301:40Dustin Getzwhy do you ask#2023-11-1301:41joshchoMainly just curious#2023-11-1301:41joshchoWondering if it was a more implementation change than anything else#2023-11-1302:19joshchoLooking into denotational design by Conal Elliott lately, perhaps for designing electric-adjacent tools (like state management). Any opinions/resources?#2023-11-1302:20joshchoHe created FRP iirc#2023-11-1302:20joshchohttps://youtu.be/rlyqoYoUumc?si=Mtflj08uUMFAzkY1#2023-11-1322:14VincentBrowsing that video briefly, his breakdowns of fn inputs/outputs is very good and something I do before making components, usually in a sketching notebook. While I see value in his delivery, I also think that monads or things that have "laws of composability" are fairly unrealistic in programming, or the kinds of programming i delve into heavily. Things do not get added equally if the sequence varies, which I find to be the key to developing fast: determining build sequence#2023-11-1414:16Dustin GetzI think a lot about denotative design and consider managed network to be a mandatory stepping stone to getting there (enabling full stack system as a function as a starting point to even thinking about these ideas)#2023-11-1414:18Dustin Getzit is extremely hard, imo it reduces to "how do we express CRUD apps in a mathematical way such that the theory of calculus, integrals, diff eqns etc (and discrete analogues) is applicable to them"#2023-11-1414:19Dustin Getzthis question is one of my personal pet topics and objectives#2023-11-1414:20Dustin Getzto summarize, i dont think the technology is there yet, except for Electric, which is closing in and perhaps making it possible for the first time to start to ask these questions#2023-11-1414:20Dustin Getzwhatever the answer is, it will be taught in high school in 50 years, like algebra#2023-11-1501:41joshcho🐐 #2023-11-1522:45chromalchemyPreach:zap:#2023-11-1701:15joshchoi spoke with conal for 3 hrs this morning and it was mindblowing. gonna keep in regular touch#2023-11-1701:21Dustin Getzhow#2023-11-1701:21Dustin Getztrip report !#2023-11-1701:21joshchojust emailed him#2023-11-1701:21joshchoyeah idk i need to rethink everything lmao#2023-11-1701:21joshchou should really talk to him @U09K620SG #2023-11-1701:22joshchoi can intro obv#2023-11-1701:44Dustin Getzhe made it pretty clear what he thinks of clojure (very negative)#2023-11-1701:44Dustin Getzwe have different goals#2023-11-1701:45Dustin Getzwhat did he say that changed your view on something?#2023-11-1702:31joshchohis interest in abstractions#2023-11-1702:31joshchoand getting to the core of the question#2023-11-1702:32joshchohe actually did not discount clj/forth or anything in our talk, he advocated for agda as the “design language”#2023-11-1702:32joshchoimplement it however u want in whichever tech stack is most fitting#2023-11-1314:25weisometimes a ui4/button will render as busy/disabled even after a hard refresh. i assume it thinks there is a remote effect still running. what are some reasons this might happen? can i clear all pending effects after a refresh?
fwiw here's the relevant code:
(defn toggle-scope [[scope-template-id size :as scope-template-pointer] added-scopes & [default-val]]
(if-let [matching-scopes (seq
(filter (comp (partial = scope-template-pointer)
(juxt :scope-template-id :size)
second)
added-scopes))]
(doseq [[scope-id _] matching-scopes]
(foreign-append! @scopes-depot
[:delete {:bid-id BID-ID
:scope-id scope-id}]))
(foreign-append! @scopes-depot
[:upsert (merge {:bid-id BID-ID
:scope-id (util/uuid)
:size size}
default-val)])))
(ui/button
(e/fn []
(e/server (toggle-scope [id size] (e/snapshot added-scopes)
(assoc DEFAULT_VAL :scope-template-id id))))
(dom/props
{:class (if (some
(comp (partial = [id size])
(juxt :scope-template-id :size)
second)
added-scopes)
"text-white bg-teal-600"
"text-gray-500 bg-white")})
(text (str size size-unit)))#2023-11-1509:30henrikIs what I’m trying to do here impossible?
(e/defn ToyRunner
[value]
(e/server (println value)))
(e/defn ToyExample
[Runner]
(e/client
(println 1)
(e/server (Runner. 2))))
;; Run with,
(e/client (Toyexample. ToyRunner))
;; Unserializable reference transfer (client side)
(e/client (ToyExample. (e/server ToyRunner))
;; Unserializable reference transfer (server side)#2023-11-1509:32henrikWhat I’m trying to do is to encapsulate named bags of server-side functionality and make them a pluggable/interchangable argument to a named bag of client-side functionality.#2023-11-1509:45henrikNot a blocker btw; I can turn ToyExample into a macro to make it work.#2023-11-1612:40Dustin Getzi see two issues here
1. electric fns/closures don't serialize and transmit (clojure's don't either)
2. electric fns are called from a single site, which means they collect their arguments to the site of the call. Here, due to this, ToyRunner is forced through a pointless roundtrip during which it fails to serialize.
As you noted - you can work around with a macro. Even without falling back to macros there is usually a simple refactor that avoids this, which is why we deprioritized it. Note that e/def is multi-sited (each site has a version of the e/def), so top level functions like these can be resolved from either site.
#1 will be fixed soon (the compiler can e.g. defunctionalize). #2 as well. This is likely coming right after differential electric (which is far more impactful and important)#2023-11-1612:47henrikOnce differential lands, I expect I would write this very differently anyhow.
Today, it’s dealing with calling various AI services and streaming back responses. I set this up (based on Peter’s advice) to be an accumulation into an atom, server-side, the value of which which is then streamed to a recipient, client-side. The client side handling turns out to be the same regardless, whereas the server-side bit varies wildly. So the goal was to bundle these into one client-side package, and many server-side packages. A macro is perfectly fine for now: it doesn’t need to be dynamic, just a bit more DRY than it would have been otherwise.#2023-11-1511:54henrikDoes :style have to be a literal?
I get this works:
(dom/props {:style {:top "798px" :left "17.125px" :position "absolute"}})
I can’t get this to work:
(let [style {:top "798px" :left "17.125px" :position "absolute"}]
(dom/props {:style style}))
Using master branch.#2023-11-1513:46henrikHere’s the problem:
(defmacro style [m]
(if (map? m)
`(do ~@(map (fn [[k v]] `(new Style node ~k ~v)) m)) ; static keyset
`(new Styles ~m)))
;; =>
(defmacro style [m]
(if (map? m)
`(do ~@(map (fn [[k v]] `(new Style node ~k ~v)) m)) ; static keyset
`(new Styles node ~m)))
Styles is not arity-1.
(e/def Styles
(e/fn* [node kvs]
(e/for-by first [[k v] kvs]
(Style. node k v))
nil))#2023-11-1518:12Dustin Getzi see, care to send a PR?#2023-11-1513:03J#2023-11-2114:25danbuneaThank you for sharing this as it contains several things that I could learn from:
1. electric + authentification
2. rama
3. polylith #2023-11-1610:16henrikIs it possible to achieve something equivalent to createPortal in Electric? (https://react.dev/reference/react-dom/createPortal)
I.e., insert some children into an arbitrary part of the DOM.#2023-11-1610:41s-olnot sure if the electric compiler supports that kind of dynamic dispatch, but the "obvious" implementation would be:
• declarePortal : creates an atom, watches and invokes its value (if truthy)
• to use the portal, set the atom to an (e/fn) that renders what you want
• in (e/on-unmount) clear the atom#2023-11-1610:44s-olor an a lower level, use electric-dom2/with?
(d/with (.-body js/document)
; portal content here
)
#2023-11-1610:44s-oldoing this to the body element might be a bad idea, but if you create a wrapper you'll be writing to from only one location maybe that works?#2023-11-1610:45nivekuil(binding [dom/node js/document.body] ...)#2023-11-1610:46s-ol@U797MAJ8M isn't that's what dom/with does?#2023-11-1610:47henrikThanks for the input. I tried out the binding approach before posting, since that’s how the root node is mounted. The compiler complains.#2023-11-1610:48nivekuilI do it in several places, but iirc it's a bit picky yeah#2023-11-1610:48s-oldom/with has some extra sauce that I don't understand, but the description suggests it does what @U06B8J0AJ needs
(defmacro with
"Attach `body` to a dom node, which will be moved in the DOM when body moves in the DAG.
Given p/for semantics, `body` can only move sideways or be cancelled. If body is cancelled,
the node will be unmounted. If body moves, the node will rotate with its siblings."
{:style/indent 1}
[dom-node & body]
`(binding [node ~dom-node]
; wrap body in a constant frame, so it can be moved as a block
(new (e/hook hook node (e/fn* [] keepalive ~@body))))) ; todo remove#2023-11-1610:48henrikdom/with sounds exactly like the ticket#2023-11-1610:48henrikGonna try it out, thanks#2023-11-1612:58henrikNo dice with with unfortunately, same error (`Failed to execute 'insertBefore' on 'Node'`)
I’ll explain what I’m trying to do: global notifcations, where they are rendered near the root of the DOM, whereas the originators sit deep in the DOM.
I see two alternatives:
1. Set up an atom, chuck up notifications as the atom is modified. Downsides: stateful, have to build an API, and customizations need to be covered by that API. Upsides: Possibly, notifications are automatically cooperative with regards to each other (hide the rest when there are more than three, etc.)
2. Find a way to portal the notfication to the notification container. Downsides: seems hard to do. Upsides: Notifications can be constructed locally, no need for an API.#2023-11-1614:18nivekuilyeah that error is what I mean by picky. I think you have to move the parent component of the "portal" around so that there's no dom elements after it or something#2023-11-1614:18nivekuilor wait a few hours, it looks like incremental is landing very soon and might just fix it outright#2023-11-1612:40mattiasWhat are folks using to manage more complex CSS setups? Is there anything Electric-specific in this regard, or just the same requirements as of a shadow-cljs project?
I'm working with a UX developer on an Electric app who doesn't know Clojure, so things like shadow-css won't work for me. We would like a bit more involved approach than sharing a huge CSS file. Something like Sass/Less would work, but I'm curious what others are using.#2023-11-1613:33telekidtailwind is honestly great#2023-11-1613:34telekidhaving tried basically everything over the last decade (css modules, css-in-js, BEM, sass/less, etc.) I can happily say that tailwind lets me just stop worrying about styles#2023-11-1613:56henrikI agree. We’re using Tailwind, and it would have to be something truly spectacular for us to choose anything else. But, all of us are also comfortable working in code.
But with Tailwind, that is true regardless of the programming language: to get leverage, your UX person needs to be in the source files.#2023-11-1614:05grounded_sageTailwind#2023-11-1621:19andersmurphyI’ve enjoyed https://github.com/green-coder/girouette#2023-11-1701:15joshchothis + electric = 🤯
https://twitter.com/multikev/status/1724908185361011108#2023-11-1800:28grounded_sageHappy to report that Quil works in Electric with zero issues I can see at present.#2023-11-2007:20HendrikOne question about the mount/unmount behaviour of electric components. Given this minimal example:
(e/defn A [x]
(println "a" x)
(println "a mount")
(e/on-unmount #(println "a unmount")))
(e/defn B [x]
(println "b" x)
(println "b mount")
(e/on-unmount #(println "b unmount")))
(e/defn Switch []
(let [!switch (atom [:a 1])
[k v] (e/watch !switch)]
(ui/button
(e/fn [] (reset! !switch [:a 1]))
(dom/text "a1"))
(ui/button
(e/fn [] (reset! !switch [:a 2]))
(dom/text "a2"))
(ui/button
(e/fn [] (reset! !switch [:b 1]))
(dom/text "b1"))
(ui/button
(e/fn [] (reset! !switch [:b 2]))
(dom/text "b2"))
(case k
:a (A. v)
:b (B. v))))
If I press a2, b1, b2, then this is printed:
a 2
b 1
b mount
a unmount
If a component does not go out of scope, but is called with different arguments, then no unmount happens. This is somewhat similiar to reacts use_effect.
B is mounted before A is unmounted. I always thought, that the unmount of the old component happens first. Is my observed output expected by the electric execution model?#2023-11-2007:28henrikAs far as I understand it, if there’s no causal connection between two units in Electric, it’s a coin toss (or at least incidental, and you shouldn’t rely on) which branch will execute first. Or you could regard it as happening at the same time, and the prints coming in a random order.#2023-11-2011:34Dustin Getzthe object lifecycle is at expression granularity really, not at function granularity, due to fine grained reactivity. react reactivity is only at function boundaries. does this answer your question?#2023-11-2011:38Hendrikyes it does. Thanks for your response 🙂#2023-11-2016:47avocadeIC in da house? :star-struck:
https://github.com/hyperfiddle/electric/commit/574a8f73df9ce57ce3c7e194b332f061cbfd7c13#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4ed
Looking promising @xifi et al 🙌:skin-tone-2: Can't wait to have faster dev compiles.#2023-11-2019:18Tommy JollyboatOooh does this indicate that it's done-and-being-tested? I opened slack specifically to check for news of IC...#2023-11-2021:54nivekuili've been using it for a few weeks now, the only bugs I've noticed are with js interop (.. field -access) not compiling and #js {:x 1} being different from #js {"x" 1} somehow. definitely worth the compile time savings#2023-11-2108:24avocade👌#2023-11-2112:03Dustin GetzIC did not pass tests and demos until like a week ago, I think @U797MAJ8M has been using regular old whole-program compilation from the IC branch, which was left in place due to breaking changes#2023-11-2112:13henrikI briefly tried the IC branch a couple of weeks back, for fun. Even though I wasn’t using the new endpoints, it did shave off 7 seconds (from 25 to 18) or something from compilation times. I have no idea why.#2023-11-2112:53Dustin GetzI believe that is what @U797MAJ8M is reporting, i don't know why either#2023-11-2116:30Tommy Jollyboat@U09K620SG thanks for the update! This may be an unanswerable question - but roughly when do you expect it to be released / merged into main? Within 2 weeks? Sometime in December?#2023-11-2116:31Dustin GetzIDK, it goes out to our design partners next#2023-11-2116:31Tommy JollyboatOK, thanks!#2023-11-2121:16nivekuilI was using the branch, switched to using -ic (when the starter app passed tests?) and now I'm testing the squash commit#2023-11-2121:18nivekuilI think those bugs I reported were only on the squash commit and not on the earlier -icnamespaces fwiw#2023-11-2121:22nivekuilI've also only been testing cljs, think we need -ic for a cljs only setup? there's e/local but it still needs a .clj file I think#2023-11-2122:45henrik@U797MAJ8M Check the WIP readme in squashed—it specifically says that CLJS only is not supported, only CLJC.#2023-11-2123:36nivekuilthat means server code can't see cljs files anymore. I can tell you with absolute certainty that cljs only (single peer no server) does work, and I think only works with incremental?#2023-11-2202:28Dustin Getzcljs only is surely possible, you may need a custom entrypoint, the default is server biased now due to some reason i don’t immediately recall, we will look into it#2023-11-2204:33nivekuil(set! reactor (((hyperfiddle.electric.impl.runtime/main Main_hf_client_client) (y/rdv) #(do))
#(js/console.log "Reactor success:" %)
#(js/console.error "Reactor failure:" %)))
is how I do the cljs only entrypoint#2023-11-2213:14xificurC@U797MAJ8M I force pushed today a fix with interop that might have been the root cause. If you have reproducible minimal repros of bugs on the ic-squashed branch that could be helpful#2023-11-2213:27henrik@xifi Do you know if the http-kit adapter is good to go? I might take it for a spin if it is.#2023-11-2213:40xificurCI don't have a repo with httpkit, so untested. In theory if you change the entrypoints as in the https://github.com/hyperfiddle/electric-starter-app/compare/ci it should work.
Note that we're still testing IC internally#2023-11-2213:52henrikI get that. It wouldn’t be to go to production on it, but to help weed out problems.#2023-11-2312:56xificurC@U797MAJ8M I couldn't reproduce but I think it has something to do with the cljs-only setup you have. I've seen the same error show up in other contexts, if I stabilize it somewhere I'll take a look. Thanks for checking!#2023-11-2102:31weianyone have an electrified drag-to-reorder implementation? figured it's worth asking before reinventing the wheel#2023-11-2112:01Dustin Getzis this for a datagrid?#2023-11-2115:30weinot specifically, but I could explore getting my data in that format if it makes things easier#2023-11-2316:02xificurCElectric bugfix release: https://clojars.org/com.hyperfiddle/electric, https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#v2-alpha-469-gb6d9865c--2023-nov-22.
This is a bugfix release before we finish our work on incremental compilation. Most notable fixes:
• too much recursion error
• websocket disconnects
Happy hacking!#2023-11-2316:04henrik<a><button></button></a>: my eyes are bleeding#2023-11-2316:48Geoffrey GaillardYes, the https://html.spec.whatwg.org/multipage/text-level-semantics.html#the-a-element says interactive content, including button is forbidden in a . Browsers accepts it anyway. We should not break browser defaults.#2023-11-2316:55henrikYes, the web has a long history of letting people do weird things for dubious reasons, in all aspects.#2023-11-2412:30joshchoPossibly a dumb question, but how do I run an event every 100ms? I tried using (int (/ e/system-time-ms 100)) but couldn't figure out how. The event does not depend on the value.#2023-11-2412:50xificurC((fn [_time] (println :event)) (int (/ e/system-time-ms 100)))#2023-11-2412:53joshchooh#2023-11-2412:53joshchoi feel dumb#2023-11-2412:53joshchoty#2023-11-2514:54joshchoHas anyone tried embedding electric app in electron? Wondering how difficult it is#2023-11-2517:45grounded_sageI built a small Swift script that spins up a native webview similar to Tauri. Very lightweight and fast. #2024-05-2110:03braai engineerHi @U05095F2K, could you show please how you spun up a Native WebView with Electric Clojure? (it's for my toy braai (barbecue) game made in Electric Clojure - see post in channel).#2023-11-2517:46grounded_sageThen you can just package the JRE and boot it from the Swift executable. But I would prefer using Graal native image and waiting for that to land. #2023-11-2709:37elkenThanks for Electric! I've been briefly tinkering around this morning on copying the #reagent demo in #uix and I'm curious if it's worth PR-ing it to the examples repo? It's much simpler since uix includes the machinery for creating and rendering to roots directly
I don't think I can easily run the examples app since I don't have datomic access, but I can submit a PR still?#2023-11-2711:24Dustin Getzsure, btw which demo are you talking about exactly?#2023-11-2711:25elkenhttps://electric-examples-app.fly.dev/user.demo-reagent-interop!ReagentInterop this#2023-11-2711:26Dustin Getzoh ok#2023-11-2711:33elkenIs the examples repo subject to the same agreement mentioned here? https://github.com/hyperfiddle/electric?tab=readme-ov-file#community#2023-11-2720:01Dustin GetzNo, I see now that we forgot to license the examples, they are MIT#2023-11-2810:29grounded_sageIf I have an atom with a sequence of data server-side and watching it from the client. What happens when I add a value. Is the whole sequence of data sent or the diff?#2023-11-2811:57henrikYou can’t watch it from the client, so I presume the e/watch is server-side, and you’re reading the resulting value client-side.
In this scenario, you’ll get the whole sequence every time it changes (modulo Electric relieving some of it).
If it goes into for-by server-side before doing something client-side in the closure of it, then you’ll get diffing.#2023-11-2813:38grounded_sageYep your clarification is correct. Thanks that makes sense with the for-by.
So if I did (e/server (e/for-by …)) on a sequence on the backend. Then when I add data to that sequence and remove data from it on the backend. The frontend will only ever receive the diff adding and removing to the rendered output?
If so this may radically simplify the design I was considering. #2023-11-2813:44grounded_sageThough I am not rendering to dom and do need the full sequence to pass to GPU at present so maybe it doesn’t quite do it. #2023-11-2813:49henrikYou for-by identity-fn, which can be as simple as a keyword, if your sequence contains maps, for example. Electric will tear down and build up branches based on that identity function.
For example, in this case the branch for 2 will be torn down (which means a small amount of communication with the client), and the branch for 4 will be mounted (which means an amount of data transferred depending on your e/server and e/client blocks inside SomeFn)
(for-by :id [m [{:id 1} {:id 2} {:id 3}]]
(SomeFn. m))
;; =>
(for-by :id [m [{:id 1} {:id 3} {:id 4}]]
(SomeFn. m))#2023-11-2813:53henrikPerhaps this is clearer:
(e/server
(e/for-by :id [m [{:id 1} {:id 2} {:id 3}]]
(let [id (get m :id)]
(e/client
(dom/div (dom/text id))))))
;; Initial payload to client: 1, 2, 3
(e/server
(e/for-by :id [m [{:id 1} {:id 3} {:id 4}]]
(let [id (get m :id)]
(e/client
(dom/div (dom/text id))))))
;; On update: <teardown instructions for 2>, 4#2023-11-2813:54grounded_sageThis is much clearer. Thank you.#2023-11-2813:54henrikI.e., only the value of :id passes the client/server border, so only that is transferred. No sequence, and no map.#2023-11-2813:55grounded_sageOh that is informative. I suspected that the whole map was transferred but it was just checking on that particular value. I would definitely need the entire map.#2023-11-2813:56henrikThen you just pass m across the border, provided it contains only serializable values.#2023-11-2813:57henrikYou will then get whole map payloads for the value of m, if anything changes within.#2023-11-2813:59henrikI.e., if your map has 1000 key/value pairs, and one changes, you will be transferring 1000 key/value pairs. As opposed to when destructured on the backend. This is an extreme example, it won’t be a problem most likely. Just highlighting the logic of what a unit of change is to Electric in this case.#2023-11-2813:59grounded_sageMy current thinking is I keep a count of the sequence and perhaps a hash. Then I could use e/for-by and accumulate it into a set on the frontend and do a count/hash check before passing all the accumulated data into rendering. But I’m unsure if this is the best way to go instead of just doing a more manual diffing process. Like batching the fetches and removing data I don’t want in the sequence anymore on the frontend#2023-11-2814:00grounded_sageMy maps are small. But the sequence is potentially large.#2023-11-2814:02henrikMy advice would be to not worry about the payload size initially. Go with as plain logic, and basic Electric code as you can. Trim it later when it actually works, and the logic has solidified by moving client/server borders.#2023-11-2814:02henrikBut definitely use for-by if you can.#2023-11-2814:02grounded_sageI haven’t worked on a solution yet as I was planning to use tmdjs but it was causing fatal errors with missionary#2023-11-2814:04grounded_sageThis is what I have at the moment. It's sending the entire sequence every change on demand with no buffering of data. So I would like to optimise this.#2023-11-2814:07grounded_sagerealise I could have shared this earlier as some grounding to the context. But yea I will go with for-by and keep it simple.#2023-11-2814:09grounded_sageI also was not so worried about payload size and more thinking about network request reduction. Like sending and receiving every data point might be silly rather than getting the next chunk at a certain point along the scrolling and just batching the request so plenty of data is there ahead of time.#2023-11-2908:13denikI believe leo did some work on this here but it hasn’t been integrated into electric yet https://github.com/hyperfiddle/electric/blob/46f9a0dc39f10afcb1fb80d6ff58c50f2ea04055/src/hyperfiddle/incseq.cljc#2023-11-2910:49grounded_sageinteresting#2023-11-2816:47henrikI’ve adapted a good 75% chunk of our app to ic-squashed (the remaining 25% will be the tricky bit).
Again, not going to production on this. But my plan is to 1) track the branch loosely to find problems early, and 2) ic has stricter requirements on the code that are mostly OK to do in the current version, so get most of the changes required back into our main and make it forwards-compatible. Preparation.
Some observations so far:
1. Wow, amazing effort. 25-45 second compile times drop to ~1 second. There’s awesomesauce in here.
2. I noticed a couple of cases where I can’t reference macros like [hello.world :refer [cat-dog]], but had to change them to [hello.world :as hello-world] … (hello-world/cat-dog).
3. A case of dom/props being unable to take a var on :style. However, destructuring {:keys [position left top]} and recreating the map {:position position …} on the style key works.
4. A few cases of things working OK on reload, but not on reboot of the app, and vice versa. Sorry for the vague language, but it’s hard to reproduce an exact case. Just that there could be glitches in the reload logic.
5. CLJS requires sometimes struggle in e/client blocks. Occasionally I’ve had to wrap them like:
(defn hello-world
[]
#?(:cljs
#2023-11-2816:54henrikOh, and the stricter requirements (and improved warnings) of IC has revealed a handful of things in our code that were just plain wrong, but kind of slipped through the cracks in our current code. Really nice improvements on communicating problems to the developer.#2023-11-2817:27grounded_sageWhat is ic-squashed?#2023-11-2817:43henrikhttps://github.com/hyperfiddle/electric/commits/ic-squashed#2023-11-2817:44henrikElectric with incremental compilation: “IC”#2023-11-2817:47Dustin GetzI have not yet deeply reviewed the IC changelog, one thing I note as missing is that reader conditional behavior i.e. #?(:clj ... :cljs ...) is completely changed#2023-11-2817:48Dustin Getzi think it now does "the expected, least surprising" thing, which is to say things like (e/client #?(:cljs ...)) can now properly guard cljs-only stuff#2023-11-2817:48Dustin Getzin Electric v2 reader conditonals would evaluate on JVM always i think, making them basically unusable#2023-11-2817:48Dustin GetzI might have some details wrong, I am not really paying close attention to this workstream yet#2023-11-2817:49henrikAh, gotcha. I assumed that “hiding” branch like that would lead to sync problems between client and server.#2023-11-2817:50Dustin Getzanother example: (e/client #?(:cljs #js {"a" 1})) will do the obvious right thing#2023-11-2817:51Dustin GetzI am not completely sure about the "hiding" issue under IC, but hiding a client-only literal i think is harmless and works#2023-11-2817:55henrikOne place I had to do this was in the uix macro we use to mount React components (it would complain that uix.dom is an unknown class). It probably would have worked OK if I wrapped the two branches locally.
I ended up proxying the functions and made them no-op server-side.#2023-11-2819:14xificurCAre you pulling the branch regularly? There was e.g. a fix on requires yesterday.
If you provide repros of the issues I might take a look at them. Order of preference is
• rcf test
• minimal repro in starter app with complete reproduction steps#2023-11-2819:34xificurCHiding a branch with a transfer will produce a runtime crash#2023-11-2820:23henrikYes, running on latest. Alright, I will try to capture as I proceed.#2023-11-2821:54nivekuil1. you have incredible patience to tolerate 45 sec compile times, I moved to ic after like 12
2. what's the exact error when this happens?
3. does dom/style work?
4. just checking, did you add the shadow build hook?
5. I think this is actually a feature#2023-11-2900:42henrik1. It’ll be great for sure, that’s a big reason why I want to be as prepared as possible. At the same time, it’s WIP and might change at any moment, so sticking with the current version is better. I’d be prepared to say that even with 45 second compile times and occasionally surprising behaviour, Electric is still better than the alternative.
2. There’s no error, it just silently doesn’t work.
3. Actually, I looked this up, I see that it’s the same problem that was fixed in main. It’ll be fixed when ic-squashed is rebased on master.
4. Yup, it’s been added.
5. I guess it depends. Simple values are easy to wrap in conditionals. More complicated things might require more sophisticated solutions, I guess. Personally, I was just caught off guard, since it wasn’t in the (WIP) changelog.#2023-11-2900:45henrik@U09FL65DK Is there a particular category of bad code I should be looking for when "Index 2 out of bounds for length 0"-style errors occur?#2023-11-2908:08xificurCcljs code should work in an e/client block, the exception I know for now is #js literals. I mean they work on cljs side but the clj reader doesn't have a tag reader for it.#2023-11-2908:09xificurCindex out of bounds runtime error is usually a symptom of bad macroexpansion, i.e. the server expanded the code differently than the client#2023-11-2908:10henrikAh, so I could be looking for CLJC files required in a #?@(:cljs … block etc. Thanks.#2023-11-2908:11henrikI’ve observed the CLJS in e/client block failing with cljs-bean. I’ll see if I can get a clean repro for you.#2023-11-2914:22avocadeThere’s no error, it just silently doesn’t work.That's probably my biggest issue with electric so far in daily practice, including its appetite for eating many/most exceptions. So I'm excited that you're focusing on this (more/better error msgs). Dev happiness ftw.#2023-11-2910:56grounded_sageI submitted the issue regarding tmdjs.#2023-11-3016:51Dustin Getzhi, when you report issues please add a clear description of the problem with screenshots of error messages etc so we can understand at a glance what is going on. for example, "i get <error message> when i <link to minimized diff which causes the issue>"#2023-11-3016:51Dustin GetzHistorically, reported issues of the form "X is broken in Electric" {X in XTDB, Datomic, Datahike, etc} have mostly been issues in X not issues in Electric and we have wasted many hours of our time tracking down issues in other people's projects#2023-11-3016:55grounded_sageI can post a screenshot of the error. I believe it only showed up in the browser console.#2023-12-0100:02grounded_sageAdded the screenshot. That is a good point with regard to cross repo bugs. I haven’t really dealt with this before and wonder if it’s appropriate to reference in an issue to tmdjs as well?#2024-01-2410:20grounded_sageI haven't created a minimum repro. But I just did an import into the electric fiddle repo and we have a new error.#2024-01-2410:26grounded_sageThis looks much more informative than the previous. Which was just missionary cancelled.#2023-11-2912:54bzrqHello #C7Q9GSHFV blob-wave
I am trying to grasp the RCF TDD workflow.
Is it the right understanding that an RCF Test Block will 'auto-run' when Code `outside the Test Block` is evaluated?
Please correct/guide/confirm.
NOTE: My current RCF setup doesn't exhibit any auto-run behavior, so I thought of checking here before digging further into my setup to check for any errors (dependencies conflict etc).
Thank you. 🙏#2023-11-2914:04telekidAfaik RCF doesn't have any kind of auto-running functionality. Tests run when you evaluate the test block. (Don't forget to enable tests with (hyperfiddle.rcf/enable!).)#2023-11-2914:43bzrqThank you for this confirmation @UABU2MMNW.
> (hyperfiddle.rcf/enable!)
Yes, it's done. :)#2023-11-3011:24Dustin Getzyes the understanding in OP is incorrect. rcf tests evaluate the same way a println evaluates #2023-11-3014:22bzrqThank you @U09K620SG.#2023-11-3010:54henrikI was looking at hyperfiddle.electric.shadow-cljs.hooks/reload-clj @xifi, should it maybe be guarded with :shadow.build/mode :dev? As per https://shadow-cljs.github.io/docs/UsersGuide.html#build-hooks.#2023-11-3012:40xificurCisn't that redundant if you only install the hook in a dev alias?#2023-11-3012:50henrikWell, shadow-cljs supports dev and release on each compilation target, so it would be more conformative to shadow-cljs standards.#2023-12-0121:21Tommy JollyboatI think I'm missing something obvious here - Using an ui4/button, something like (e/defn Thing [db] ,,, (ui4/button (e/fn [] (e/server (side-effects!) nil)) ,,,)), the button continues firing and running side-effects! recursively until it (eventually) causes itself to be unmounted. Is this because side-effects! mutates the db that's passed to Thing, causing it to re-mount? and/or because it's in some kinda "pressed" state that the callback fn could disable?#2023-12-0122:38xificurCMutating the db creates a cycle. Search for e/offload in the channel for previous discussions#2023-12-0123:26Tommy JollyboatYes! e/snapshot worked for me - I'll use that as a workaround until UI5 obsoletes it.
I don't understand e/offload, on an ELI5 level, does it effectively run something asynchronously? (while letting you optimise for the type of bottleneck you expect)#2023-12-0210:48Tommy JollyboatDon't worry, I found a quote: "`e/offload` moves a blocking computation to a threadpool (see docstring), so as not to block the reactive computation. databases are slow and blocking (even transact) so all database operations must be offloaded" 👍#2023-12-0212:05henrikI don’t see how offload prevents cycles without combining with snapshot. Or if it does, it raises other questions, like how come reactive updates to queries using db as input works?#2023-12-0218:03xificurCI meant to write e/snapshot. Shows why replying at night is not a great idea#2023-12-0205:29Drew VerleeWhats the best way to get started with hyperfiddle being backed by datomic cloud?#2023-12-0206:05VincentProbably gut the XTDB demo to use datomic instead#2023-12-0212:55Dustin Getzyou will need to adapt the datomic client core async api to missionary, which we have a first pass of in a contrib namespace, which is used in the datomic browser demo. i would start there#2023-12-0212:57Dustin Getzdevops wise, regarding Datomic Ions, due to the challenges getting a websocket to the Ion app cluster, i don’t know of anyone who has attempted this with electric, there are several threads summarizing the situation and next steps #2023-12-0212:58Dustin GetzIons is of course optional, the other configuration should work ootb#2023-12-0606:22Drew VerleeThanks Dustin.#2023-12-0219:34Christian DeanCan electric replace reagent? Still in my first year of programming so trying to understand the electric readme. "Full reactive" sounds like it can do what reagent does#2023-12-0310:44Christian DeanAwesome, thanks. I'll give it a go#2023-12-0514:25danbuneaNewbie question here. I have a list and an item (row) components. I need to pass a way (fn) to the item to mark itself as selected when clicked. This is how I tried:
(e/defn Entity-item [id name on-select]
(dom/div
(dom/a
(dom/on "click" (e/fn [_] (on-select)))
(dom/text id))
(dom/text name)))
(e/defn Entity-list []
(let [!state (atom {})
state (e/watch !state)]
(e/server
(e/for-by :db/id [item (pull-table (get-db) :a.o.r.chat.pl {:user/id 6})]
(let [id (:db/id item)
name (: item)]
(Entity-item. id name #(swap! !state assoc :selected id))
...#2023-12-0514:35henrikThe e/server closure reaches all the way into Entity-item. Wrap (Entity-item. id name #(swap! !state assoc :selected id)) with e/client.
Secondly, change #(swap! !state assoc :selected id) to an e/fn and stick e/server inside of it.#2023-12-0514:36henrikAnd then you need to call it with (on-select.)#2023-12-0514:41henrikAlternatively,
(e/defn Entity-item [id name OnSelect]
(e/client
(dom/div
(dom/a
(dom/on "click" OnSelect)
(dom/text id))
(dom/text name))))
(e/defn Entity-list []
(let [!state (atom {})
state (e/watch !state)]
(e/server
(e/for-by :db/id [item (pull-table (get-db) :a.o.r.chat.pl {:user/id 6})]
(let [id (:db/id item)
name (: item)]
(Entity-item. id name (e/fn [_]
(e/server
(swap! !state assoc :selected id)))
...))))))
I.e., e/client in Entity-item.#2023-12-0514:45danbuneaHi @U06B8J0AJ thank you very much. I wrapped the entity-item with a e/client which I shouldn have from the start. Thank you:
(e/defn Entity-list []
(let [!state (atom {})
state (e/watch !state)]
(e/server
(e/for-by :db/id [item (pull-table (get-db) :a.o.r.chat.pl {:user/id 6})]
(let [id (:db/id item)
name (: item)]
(e/client
(Entity-item. id name #(swap! !state assoc :selected id)))
#2023-12-0514:47henrikAh, so !state is on the client, I thought it was server-side.
Generally helpful to future you (and incidentally good preparation for future versions of Electric) is to start your functions with e/client or e/server.#2023-12-0515:00danbuneayes, adding them definately simplifies debugging#2023-12-0515:00danbuneaand yes, client side#2023-12-0518:58Vincenti'm very intrigude by your key name :a.o.r.chat.pl#2023-12-0518:59Vincentthe luxurious days of paying no attention to function definition sequence will soon be behind us wistful nostalgic tears#2023-12-0519:07danbunea🙂 it's actually something well crafted, it's not just a random thing#2023-12-0602:47joshchoHow would streaming videos* work in electric? Websockets are used a lot for videos iiuc#2023-12-0603:29Cora (she/her)websockets in videos?? I've never heard of that#2023-12-0603:29joshchoWhat’s usually used?#2023-12-0603:29joshchoI meant streaming actually#2023-12-0603:30joshchoLike twitch#2023-12-0603:30Cora (she/her)HLS and DASH are the big players#2023-12-0603:30joshchoI am talking to some mux ppl#2023-12-0603:30joshchoInteresting#2023-12-0603:32Cora (she/her)I worked at Zencoder which is where the people from Mux came from#2023-12-0603:32joshchoWoah!#2023-12-0603:32joshchoVery cool!#2023-12-0603:32Cora (she/her)yeah, they're nice folks.#2023-12-0603:32joshchoSo idk if u heard but twitch is leaving korea#2023-12-0603:32Cora (she/her)I didn't hear, no#2023-12-0603:33joshchoSo I am thinking about a metaverse-oriented streaming platform to fill that gap#2023-12-0603:33joshchoFor like 50 streamers I know well initially#2023-12-0603:34Cora (she/her)does anyone use metaverse?#2023-12-0603:34Cora (she/her)and why is twitch leaving korea?#2023-12-0603:35joshchoThere’s like 200k viewers at the very least#2023-12-0603:36joshchoIn korea#2023-12-0603:36joshchoTwitch is leaving bc of regulation costs#2023-12-0603:36joshchoVtubers*#2023-12-0603:36Cora (she/her)well, best of luck with that. the Mux folks should either know how to do it or know how to figure it out#2023-12-0603:38joshchoI’m talking to paul klein rn#2023-12-0603:38joshchoWho sold streamclub to mux#2023-12-0603:39joshchoTy!#2023-12-0603:39Cora (she/her)I haven't really followed Mux much, didn't know they did acquisitions, only heard about layoffs a few times#2023-12-0603:40joshchoAh gotcha#2023-12-0609:20danbuneaNewbie question no #2.
I want to render a list of entities (Entity-item) which i read from the server (pull-table).
When I edit, add or delete one I change the data on the server, but I'd like to only load the changes not the entire list so I don't want to reinstantiate the component. What are my options?
(e/defn Entity-list []
(let [!state (atom {})
state (e/watch !state)]
(e/server
(e/for-by :db/id [item (pull-table )]
(let [id (:db/id item)
name (:a.o.r.chat.pl/name item)]
(e/client (Entity-item. id name)))))))
I had the impression that swapping the client !state atom would be enough, but it's clever and doesn't do the server call again, so I need another approach.#2023-12-0610:33henrikDo I read your code correctly, that the for-by doesn’t depend on the state. Or is it an input to pull-table?#2023-12-0610:33danbuneait doesn't#2023-12-0610:33danbuneaI was just thinking to make it depend on it, then it will be retriggered#2023-12-0610:34henrikAh, OK. Yes, only things that depend on state will be refreshed.#2023-12-0610:34danbuneathis is very neat 🙂#2023-12-0610:35danbuneathank you, I am making it work now#2023-12-0610:40danbuneaworking, of course, as it all makes sense 🙂#2023-12-0610:46danbuneaso, maybe for others:
(e/defn Entity-list []
(let [!state (atom {:ver 0})
state (e/watch !state)
ver (:ver state)]
(e/server
(e/for-by :db/id [item (pull-table ver)]
(let [id (:db/id item)
name (: item)]
(e/client (Entity-item. id name)))))))
when ver changes, the pull-table will be rerun and only the changes sent to the client and rendered.#2023-12-0615:07avocadeAfter having used the ic experimental branch as the base for some time, it feels pretty much as snappy as when working with uix before. The time to recompile went down from ~40s to ~4s. So an order of magnitude improvement, not something to take for granted. Well done 👍#2023-12-1204:54Piotr RoterskiI've just moved to ic as well. It works great. ⚡
However, I've found another minor breaking change that needs to be updated in electric-starter-app:
hyperfiddle.electric-client/VERSION var passed in as closure-defines to shadow-cljs during build needs to be renamed to hyperfiddle.electric-client/ELECTRIC_USER_VERSION
Here's a PR: https://github.com/hyperfiddle/electric-starter-app/pull/16#2023-12-0807:02Nathan SmithHas anybody implemented a language server with electric? Does anyone want to help write one? 🙂#2023-12-1113:10Dave MaysThis new runtime offers local-first cross platform with "any" back-end that can communicate via std-out. I'm not smart enough to package this, but I'm wondering if this would open up the ability to run electric as a local first desktop and even mobile app? https://socketsupply.co/#2023-12-1115:49nivekuilidk what this does (found the HN thread which is hilariously astroturfed https://news.ycombinator.com/item?id=38581754) but you can already use electric just as a js library, so the usual tools like nw.js should just work#2023-12-1911:48Dave MaysAh I did not know you could use electric as a js library, that's nifty.#2023-12-1116:02nivekuildoes anyone have an example of using m/group-by with electric directly?#2023-12-1116:03nivekuil(e/for [process (new (->> (m/seed ["apple" "aardvark" "bat" "bar"])
(m/group-by first)))]
(dom/text (new (m/reductions conj process))))
some code I was playing around with, no idea how it works. I thought group-by would return a [key mapentrycompel it is, or I could fork another thread inside of offload, I guess. We use virtual threads, so the overhead is minimal.#2023-12-1315:06henrikI’m getting this on release compilation (not dev), using the latest Clojars release. I’m not sure how to approach this, please advice.#2023-12-1410:30henrikThis is intermittent, and rerunning will occasionally make it compile without errors. We’ll try to live with it until we’re on IC.#2023-12-1414:14Dustin Getzis it https://github.com/hyperfiddle/electric/issues/11#2023-12-1414:23henrikYeah, could be, although we run with Xss4m#2023-12-1412:37weiThanks for shipping IC, excited to try it out! Is there a migration path from non-IC, or is it best to clone the starter project? Would be great to be able to migrate existing projects if possible.#2023-12-1415:51danbuneaFrom @U09FL65DK :
"Take a look at this https://github.com/hyperfiddle/electric-starter-app/compare/main...ic?w=1 on how to upgrade.". This is what I did. The thread is: https://clojurians.slack.com/archives/C7Q9GSHFV/p1701979730768789#2023-12-1418:36denikusing httpkit I always get this error on boot
clj꞉dash.serve꞉>
; Thu Dec 14 13:34:53 EST 2023 [worker-4] ERROR - handle websocket frame org.httpkit.server.Frame$TextFrame@990243e
; java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
; at hyperfiddle.electric_httpkit_adapter$handle_electric_ws$on_receive__80522.invoke(electric_httpkit_adapter.clj:79)
; at clojure.core$partial$fn__5920.invoke(core.clj:2641)
; at org.httpkit.server.AsyncChannel.messageReceived(AsyncChannel.java:153)
; at org.httpkit.server.WSHandler.run(RingHandler.java:187)
; at org.httpkit.server.LinkingRunnable.run(RingHandler.java:154)
; at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
; at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
; at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
; at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
; at java.base/java.lang.Thread.run(Thr @
but everything boots fine and the app works!#2023-12-1418:39Dustin Getzis this a hot code reload#2023-12-1420:10nivekuil(when (and falsey maybe) ...)
when maybe changes, is this expression short circuited or is still a diff happening?#2023-12-1420:40Dustin Getz#2023-12-1420:40Dustin Getzit will macroexpand to nested ifs, thus matching the behavior of electric-if#2023-12-1420:40Dustin Getzi.e. untaken branch is not mounted#2023-12-1420:41Dustin GetzAre you observing something different?#2023-12-1420:42nivekuilno, just thought it was interesting to know. mounting is actually fairly expensive too right, it's the only time missionary allocates?#2023-12-1420:42Dustin Getzelectric let compiles to missionary allocation#2023-12-1420:42Dustin Getzmounting is expensive in the network planner though#2023-12-1420:43Dustin Getzi believe all mounts are currently broadcast, because the frame could contain client and server regions, and we don't yet do the network equivalent of dead code elimination (it has not yet been an issue in practice, we do measure)#2023-12-1420:45nivekuilsorry i'm tired and can't parse "electric let compiles to missionary allocation" -- what is electric let? how can you allocate at compile time#2023-12-1420:46nivekuilyou mean the array sizes are known ahead of time?#2023-12-1420:46Dustin Getz(let [x (F. a) (vector x x)) injects a m/signal! so that flow x is reused; i.e. it allocates a memo buffer in the reactor heap to store the latest value of x#2023-12-1420:49nivekuilthat happens on boot or component mount#2023-12-1420:55nivekuilwell compilation is by def now so it couldn't be the former. I was thrown off by "electric let", but that's the same let the and macro uses#2023-12-1421:18Dustin Getzm/signal! instruction is compiled into the target missionary DAG (in fact m/signal! is the operator that makes it an actual DAG as opposed to a tree, you need the memo buffer to get a DAG). The allocation happens at runtime. I believe the exact details of how at runtime it happens (i.e. "when the flow boots") was the subject of the missionary DAG supervision release, which supersedes m/signal! with m/signal and improves the semantics here. Electric v2 still uses the old way, the differential rewrite uses the new way#2023-12-1510:25henrikIve got a spinner. The entry point for it looks like this:
(e/defn MountSpinnerAs
[id]
(e/client
(let [set-loading-state! (second loading-state)]
(ops/when-repl (timbre/info "Mounting spinner for" id))
(set-loading-state! #(conj % id))
(e/on-unmount (fn []
(js/setTimeout
(fn []
(ops/when-repl (timbre/info "Unmounting spinner for" id))
(set-loading-state! #(disj % id)))
60))))))
I’m calling it here:
(e/defn BindAccount
[Next]
(e/server
(let [{:keys [logout-url]
:user/keys [account]
:as user-info}
(try (session/Authenticate.)
(catch Pending _ nil))]
(if-not account
(MountSpinnerAs. :BindAccount)
This generates the following error:
TypeError: Cannot set properties of undefined (setting '3')
at Object.hyperfiddle$electric$impl$runtime$input_change [as input_change] (runtime.cljc:373:15)
at hyperfiddle$electric$impl$runtime$eval_change_inst (runtime.cljc:819:6)
at eval (core.cljs:7387:25)
at Object.cljs$core$inode_kv_reduce [as inode_kv_reduce] (core.cljs:7391:28)
at Object.eval [as kv_reduce] (core.cljs:7633:6)
at Object.eval [as cljs$core$IKVReduce$_kv_reduce$arity$3] (core.cljs:8197:50)
at Object.cljs$core$_kv_reduce [as _kv_reduce] (core.cljs:719:16)
at Object.cljs$core$reduce_kv [as reduce_kv] (core.cljs:2628:8)
at hyperfiddle$electric$impl$runtime$parse_event (runtime.cljc:836:11)
at eval (core.cljs:5750:36)
This is fixed by adding e/client:
(e/defn BindAccount
[Next]
(e/server
(let [{:keys [logout-url]
:user/keys [account]
:as user-info}
(try (session/Authenticate.)
(catch Pending _ nil))]
(if-not account
(e/client (MountSpinnerAs. :BindAccount)) ;;
Why? (This is IC)(e/client ..) now at the top of electric defns it looks like.
Exciting stuff.
I have two apps that use electric, I am building a new one using Rum mainly because I need good SEO and therefore serverside-rendering for it. But I am still dreaming up ways I could "drop in" Electric as a live component in a larger application, like how the one group did their calendar/scheduling selector as a multiplayer iframe.#2023-12-1713:39JHi! I start to use rcf (very cool by the way!) and I wonder if I can require namespace inside the tests like this:
(rcf/tests
(require '[malli.generator :as mg])
"Gen some date"
(mg/generate traffic-abacus-schema))))
Unfortunately kondo give me an unresolved-namespace on mg/generate. It's ok to do this and put a condition to tell to kondo to ignore unresolved-namespace inside tests or require the namespace in the ns directly?#2023-12-1713:56borkdudeYou could also put the require in a comment before the RCF form #2023-12-1713:56borkdudecomment form#2023-12-1719:55noonianDoes e/local still work with the incremental compilation changes on master? I'm seeing an error check failed: (some? (get (aget frame frame-slot-dynamic) symb)) for nil but it could be user error on my part#2023-12-1808:32xificurCIf you're writing tests we switched to l/local, https://github.com/hyperfiddle/electric/blob/89649d65902bfea8986d484aa4a281968646fb86/test/hyperfiddle/electric_test.cljc#2023-12-1821:45noonianHi, thanks for the response. I was previously using e/local as an alternative to the older e/boot for client-only apps without a server component. Is that something that is supported with IC or will be in the future?#2023-12-1908:47xificurCwe don't have an off-the-shelf setup for that, but the https://github.com/hyperfiddle/electric/blob/89649d65902bfea8986d484aa4a281968646fb86/src/hyperfiddle/electric_local_def.cljc contains a single+ which seems to be a drop-in replacement for what you describe#2023-12-1721:40Dennis OrsiniFrom a previous thread by @petrus it was mentioned by @leonoel that IC was the main technical blocker for AOT compilation via Graal native-image. Since IC has landed in master, I wanted to follow up and know if that was still the only blocker and if there are any ideas about running the https://www.graalvm.org/latest/reference-manual/native-image/metadata/AutomaticMetadataCollection/ from the electric entry point to try and get some workable binaries out. For context, I haven’t tried any of this with electric, but I am willing to run some experiments if anyone can provide some pointers or some more context on what else might be missing. Otherwise, love the work you all are doing and happy holidays.#2023-12-1808:35xificurCcan you link to the previous discussion?
We made no efforts on this front, nor have any immediate plans. You're welcome to try though! IC gets us closer to typical clojure/script compilation#2023-12-1809:00Dennis Orsini@U09FL65DK https://clojurians.slack.com/archives/C7Q9GSHFV/p1689156739495209#2023-12-1809:01Dennis OrsiniAnd the reply https://clojurians.slack.com/archives/C7Q9GSHFV/p1689170129268449?thread_ts=1689156739.495209&channel=C7Q9GSHFV&message_ts=1689170129.268449#2024-06-2418:30grounded_sageI would like to point out that a nice benefit of GraalVM compilation is you can load your Electric app into a Tauri application using the sidecar functionality. So you could build "native apps" with Electric. Also acknowledge this very edge case benefit.#2024-06-2418:58braai engineerwould love to turn my BraaiSim game into a native app “for free” but first gotta figure out how to make a backendless (client-only) Electric app first. I understand that it is possible.#2024-06-2506:33braai engineerClient-only is hostable on static hosting, so cheap scale and near-instant deployment. But yea braaimaster high scores and stuff need backend.#2023-12-1803:04joshchoI am having trouble updating the content of ui/input while I am focused on it. When the input becomes "\lorem", I want to replace the input with "Lorem ipsum dolor sit amet, consectetur adipiscing elit.".
(let [!txt (atom "")
txt (e/watch !txt)]
(div (text txt))
(ui/input
txt
(e/fn [v]
(if (= v "\\lorem")
(reset! !txt "Lorem ipsum dolor sit amet, consectetur adipiscing elit.")
(reset! !txt v)))))
The change propagates only after I focus away from the input. How would I solve this issue?#2023-12-1808:37xificurCThe UI controls do this on purpose, otherwise a concurrent update (e.g. from another user) could flow in as you're typing and make a mess#2023-12-1810:39joshchoI see. Is there a way to resolve this problem? This is a core feature in my app#2023-12-1810:42xificurCif the strategy bundled in ui controls doesn't work for you you can roll your own with dom/input#2023-12-1810:58joshchoah right right#2023-12-1816:56Dustin GetzDifferential Electric – very early sneak peak of streaming dom rendering over a large collection. Scenario is an industrial robot (shipyard crane) which we are observing live by tailing a log file. The robot runs at 20hz which the UI tails in realtime. The tracks are a plot of various record fields from the robot log, if you look closely you can see they are waves because the robot is moving back and forth. The tracks here are rendered in SVG and there are 1500 SVG elements being maintained (which is the soft limit according to google).#2023-12-1821:52nivekuilwhat is that fps monitor?#2023-12-1821:55nivekuilwhat if you weren't streaming logs but something mutable?#2023-12-1821:57Dustin Getzfps display is chrome#2023-12-1821:58Dustin Getzwhat is the mutable thing you have in mind to stream?#2023-12-1822:01nivekuillike if you're watching files instead of tailing logs, does differential have any impact on how you'd do it?#2023-12-1822:02Dustin Getzyou’ll need to snapshot right? values over network#2023-12-1822:03nivekuilsay you knew the memory layout of the robot, to read the state you map the memory into a struct, and then how does it go over to electric? have to write your own diffing protocol still (to do it efficiently) or does electric have an opinion on that#2023-12-1822:06Dustin Getzin this case a crane connector is translating the hardware structs and appending to a log; the structs arrive over a socket, the crane firmware is responsible for snapshotting the proper memory offset and writing to socket #2023-12-1822:13nivekuilso in your case you start with a discrete flow (socket), pre-diffed? what if you started with a continuous flow (file) too large to send over the network, does differential do/prefer anything special?#2023-12-1822:15nivekuili'm guessing you'd still have to manually partition the flow into smaller flows for more efficient reactivity, differential doesn't help with that?#2023-12-1914:38Dustin Getzyes, you'll need to at least inject a diffing keyfn, i.e. imagine e/diff-by (i.e. the first stage of e/for-by)#2023-12-1914:39Dustin Getzif you are ingesting a batch data source like a file or sql resultset you probably just want e/for-by , which under Differential does the same thing as it did before just fancier internals#2023-12-1816:58Dustin Getzin this demo, halfway through when we speed up the playback rate, you can see the DOM stream in at the edges#2023-12-1817:04Dustin GetzOne more, here we stream in 8k dom nodes basically instantly (note there are 3 tracks, the first one is faint)#2023-12-1822:20denikis this dev or prod? perf is very different local vs with network latency#2023-12-1822:56Dustin Getzthis app runs on a LAN, but to your broader question, there aren’t any round trips so if network adds delay i don’t expect we would have a way to notice#2023-12-1904:23denikmakes sense#2023-12-1906:00denikwould it be more efficient to render all dom nodes at once as react tends to do?#2023-12-1907:21henrikI don’t know if every dom node corresponds to a network event or not, and whether they are logically a sequence, but if there are 8000 network events, the network latency kind of only applies to the first one. The first event to arrive will be delayed by 10/20/40/whatever ms, but after that they would arrive in roughly the same frequency as they were sent. Network latency is more of an “offset” than anything, when talking about bursts of events. Given a relatively stable internet connection.
This changes if each requires an ack, which isn’t the case here.#2023-12-1908:37avocadeCan't wait#2023-12-1917:37denik@U06B8J0AJ good point!#2023-12-1917:37denikif the payload is streamed#2023-12-1917:46denikwhich is being explored here https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/incseq.cljc#2023-12-1917:49Dustin Getzthis demo is implemented with incseq actually, incseq is the work horse of electric v3, the rest is mostly just syntax and upgrading the electric runtime to use incseq operators everywhere instead of vanilla missionary#2023-12-1917:53denikvery exciting!#2023-12-1818:15grounded_sageAmazing#2023-12-1909:51henrikIs the low-level “peer” abstraction new, or were client/server always concretions?#2023-12-1921:06xificurCwhat do you mean by "peer" abstraction?#2023-12-2007:50henrikThe caveat here is that I haven’t read the existing Electric codebase thoroughly, and my understanding of the particulars is blurry.
I’m following along the DE commits out of interest, and I don’t think I’ve seen the words “server” and “client” yet, only “peer”, which reminded me of Dustin saying that there’s nothing in the idea of Electric, fundamentally, that prescribes a server and a client, it could be peer to peer. So I was wondering if you’re paving for that road as well In this effort.#2023-12-2008:57xificurCour target is 2 peers and will be for the near future. In theory the 2 peers can both be clj or cljs (some tests already do this). In theory you could replace server/client with foo/bar. Not on our DE radar though#2023-12-2009:51henrikI guess going beyond that is to start walking on consensus algorithm territory#2023-12-2016:34avocadeInteresting thoughts henrik 👍#2023-12-2018:45nivekuil2 cljs peers could be very nice for web workers#2023-12-1918:51denikis the cljs :require-macros expr still needed in ic with the shadow reload hook?#2023-12-1921:05xificurCif you refer to the hack we had where every electric file needed a :require-macros on top, that hack was removed months before IC. Maybe we didn't communicate it clearly#2023-12-2003:51denikyup that was it#2023-12-2021:05denikwhen an electric file requires other electric files, does it need a require-macros expr for cljs or are regular requires all that’s needed now?#2023-12-2021:07xificurCwe no longer rely on require-macros for any electric-specific behavior, i.e. you should use it when it's needed in regular cljs, requiring macros from another namespace#2023-12-2021:07denikhaving issues where a reactive value triggers a dom listener more than once which is undesirable. I already tried using snapshot to avoid this but that results in stale data. Is there any way to guarantee a function only runs once on invocation? here’s the listener code#2023-12-2021:08denik#2023-12-2021:11Dustin GetzWon't "running function only once" also result in stale data?#2023-12-2021:18denikonce per dom event#2023-12-2021:21Dustin Getzmaybe model this as a missionary flow for direct control? dom/on is the primitive we are unhappy with for nontrivial situations#2023-12-2021:21Dustin Getz#2023-12-2021:23Dustin GetzI think what you want can be achieved with e/snapshot but it's definitely complicated and dumb#2023-12-2021:24Dustin Getzif it can't, I don't see why#2023-12-2108:04denikwill try the above. I wonder if because the value (`cell-ent`) in my case is a e/watch(ed) value electric would rerun even the missionary code#2023-12-2108:06denikwhen I tried snapshot it resolved the repeated runs of the dom handler at the price of stale data#2023-12-2109:17xificurCyou can also snapshot higher, e.g. the whole vector passed to db/transact#2023-12-2120:18denikfinally boiled down this issue! https://github.com/hyperfiddle/electric/issues/64#2023-12-2120:18denikit does look like a bug / unintended behavior to me#2023-12-2022:21Dustin GetzI'm planning our content schedule (blogs and talks) for the spring. Yes, we'll be talking about Differential Electric. Besides that, what else do you want to hear about?#2023-12-2102:38Vincentauthentication strategies would be helpful for me#2023-12-2112:08danielnealI know you’re focussed on back office apps but I was very impressed with the industrial robot demos and I’d be interested to hear about whether electric (especially with differential) is is still only suitable for back office things or if it might be suitable for more front-end / client-facing use cases#2023-12-2115:43Adrian SmithOptimistic updates, missionary documentation website#2023-12-2118:55grantI’d second @U055PQH9R4M’s authentication request. That is not the sort of thing you want people to have to get right on their own, if it can be avoided.#2023-12-2216:36chromalchemyIm interested in fine grained state construction and manipulation to do bret victor stuff.
This is maybe less about the network if not doing multiplayer, but im interested to see strategies from electric context (ie, is an atom enough? Need something like electric datascript or rules engine, etc?)#2023-12-2301:47m pheasantI'm just getting started with this, and love the idea in the always-online scenario.
I'm interested to hear more about the intermittent-online possibilities.
I'm about to try out scenarios to see how losing connectivity is handled currently - I understand that there are upcoming plans for this - maybe more info on this would be of wider interest?#2024-01-1301:48Daniel JompheLate here, but yes, what about choppy wireless connectivity, is resiliency for this on par with regular frameworks, or is there some more automatically done for users?#2023-12-2110:16CanHello! I'm currently working with the history router and was wondering if there's a way to configure it to open links in a new tab, similar to the "_blank" attribute in traditional anchor tags. Is this functionality supported, and if so, could you provide a code snippet to help me better understand how to implement it?#2023-12-2113:23Geoffrey GaillardShould do the trick:
(history/link <target route>
(dom/props {:target "_blank"})
(dom/text "Click me"))
#2023-12-2113:27CanThank you so much!#2023-12-2113:40CanIn the context of the history router, if I'm using the history/navigate! function with a route like
(history/navigate! history/!history [:app.poms-00-main/entity-info-page [company-name "company"]])
how would I open in a new page?#2023-12-2113:51Geoffrey GaillardAn history instance is local to a tab.
For now, if you want to open a new tab programmatically:
• get the link from the route with: (history/encode (history/build-route !history <route>))
• pass it to https://developer.mozilla.org/en-US/docs/Web/API/Window/open#2023-12-2206:51CanUsage example:
(.open js/window (history/encode (history/build-route !history [:app/targetkw [param1 param2]])) "_blank")
It's working.#2023-12-2113:25Geoffrey GaillardNew easier, self-contained way to get started and experiment with Electric: https://github.com/hyperfiddle/electric-fiddle#2023-12-2116:37vollcheckhi, I've ran through the instructions in README and spotted couple typos - https://github.com/hyperfiddle/electric-fiddle/compare/main...vollcheck:electric-fiddle:main#2023-12-2116:37vollcheckotherwise great repo and happy to play around with it, I appreciate that every new iteration brings more "hygiene" to the table#2023-12-2117:06Dustin Getzthanks @U03N5HN4K1N - So that you get credit for the contribution, can you create a PR and comment on the PR that you release this into the public domain? (We still have to select a license for this repo)#2023-12-2118:41vollcheck@U09K620SG here it is: https://github.com/hyperfiddle/electric-fiddle/pull/1#2023-12-2214:45tobias Is this a replacement for https://github.com/hyperfiddle/electric-starter-app or does it serve a different purpose?#2023-12-2214:49Dustin GetzYes that is the intent though we haven't made a final decision about what to do about that repo#2023-12-2203:49JAtkinsI may have just missed it somewhere, but how can an electric user avoid the danger of remote code execution vulnerabilities? Especially with the dynamic nature of CLJ, it seems rather easy to get electric on the server to call any function with any args. Even if at compile time the list of acceptable fns is constrained to what the electric compiler identifies as existing fn calls, the api size is far larger than with a normal rest api.#2023-12-2211:24Dustin Getzserver is fully determined and baked at compile time as of the new incremental compilation changes. not just the list of functions but also the access patterns#2023-12-2217:38JAtkinsso, each unique context change from e/client to e/server is named and identified? cool, that sounds good.#2023-12-2217:59Dustin Getzyes the electric analyzer slices the system DAG in half, the cut edges are pub/sub (input/output) pairs with addresses used by the wire protocol#2023-12-2210:54joshchoI am trying to get video working via dom/video, and I can't fast-forward. Can't distinguish if this is an Electric issue or something else.
(dom/video (props {:controls true :width "100%"})
(dom/element :source
(props {:src "southwell1.mp4"
:type "video/mp4"})))#2023-12-2214:21Dustin Getzif you suspect an electric interaction, you can always drop to vanilla js like (doto (js/document.createElement "video") (.appendChild dom/node) (.setProperty "src" "southwel1.mp4") ...)#2023-12-2214:21Dustin Getz(pseudocode)#2023-12-2214:50Geoffrey GaillardIt's not electric, it's jetty / ring.
Fast forward requires the web server to support https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests. IIRC there's a ring middleware for that.
S3 / CloudFront supports it ootb, also IIRC#2023-12-2218:24Dag NorbergI encountered the same thing, check this:
https://github.com/formicagreen/electric-media-player#2023-12-2223:56joshchowhat is the best way to query for current time of the video as a missionary flow/electric value?#2023-12-2300:00joshchoif one can point me to any similar resources, that'd also help#2023-12-2219:02Dag NorbergSome thoughts and questions…
Apologies if none of this makes sense… Not an expert developer, just a curious person :)
Apart from the whole frontend/backend transparency, one thing that makes Electric so nice is it lets you think of your app sort of like a spreadsheet.
Ideally this would also be true for React et al, but in reality you end up with a lot of accidental complexity, lifecycle management etc. Less so with Electric.
This made me think about “computed values” in a spreadsheet. This is sort of like a pure function operating on the db.
My question is if Electric could “solve” cache invalidation for these kinds of values?
Let’s say you have an “order” item that contains a bunch of product ids. And you want to show some compound information about the order and products.
Normally you have to choose between:
• Recomputing on every request
• Caching the results, and having to come up with a strategy for invalidating the cache
Could Electric potentially do this in an “ideal” way? I.e. you set a watcher on the db and updates “trickle down” to wherever they are needed?
I’m not talking about frontend/backend sync here, just cache invalidation in general.
Another example is a CMS / static site generator. Most of these have very simplistic models for how cache invalidation / partial regeneration are done. But on a large site, content may be interweaved in all kinds of ways, and predicting what pages are affected when a piece of content is updated becomes more complex.
Taking this a step further, could you implement a kind of “event sourcing” system purely in Electric? E.g. “send email on new order”.
I have been reading about Rama and from my limited understanding this is somewhat similar. But it seems difficult to learn and a bit overkill for the type of work I’m doing. I like that Electric looks like regular Clojure code.#2023-12-2219:25Dustin GetzYes, most of this is already true in Electric v2, and with Differential Electric becomes fully true. With Differential Electric, instead of cache invalidation + recompute, you propagate diffs through the system and each "cache" (view) is incrementally maintained#2023-12-2219:25Dustin Getzthe database watcher, if the database supports it, is the source of diffs (along with any other user events, atoms or sources of variability)#2023-12-2219:31Dustin GetzRe. Rama, Nathan is the right person to be building that, we have no interest in hyperscale backends, Electric is exclusively a UI technology, BYO data tier. Nonetheless, sure, hypothetically an Electric backend can "send email on new order", in the abstract this is the same as "add dom element for new order", however the devil is in the details. Rama offers a bunch of strong operational guarantees (like you'd get from a database) which Electric does not#2023-12-2309:45Dag NorbergThanks for the explanation!#2024-02-0710:20Dag NorbergHi, reviving this... thinking about the example again of a cms / static site generator.
Would it be a viable to use electric (+ datomic or something) to generate a bunch of html files and minimally regenerate stuff when the database is updated? I.e. instead of "update dom when x y or z changes" it's "spit html file". Or would there be similar issues with operational guarantees?
It seems like incremental maintenance + effects would be incredibly powerful in a bunch of contexts outside of ui. Like being able to model your entire app as a reactive system. Or are there established solutions for this that I'm not aware of? Again Rama seems very cool but I have no need for that kind of scaling.
Another thing, do you think differential electric could potentially fix the perf issues I had with the drawing app?#2024-02-0711:07Dustin Getzyes, differential will fix some of the Painter app perf issues (the slowdown with eg. 1000 objects). the local first update issue will become possible to fix as well but will require a bit more work #2024-02-0711:09Dustin Getzyes, i think using electric to render and maintain static html files on disk should work in principle - you may hit edge cases though that need smoothing out#2024-02-0714:43Dag Norbergthanks again for taking the time to answer! really cool following this stuff#2023-12-2219:46denikis there a way to ensure an that a e/defn flow invoked from within a dom handler completes? It currently appears that something refreshing in the UI cancels the flow which in turn cancels necessary DB transactions#2023-12-2219:50Dustin Getzah, the rule here is: the e/fn is unmounted when the first non-Pending value is seen by the client#2023-12-2219:53Dustin Getzone of your snippets showed basically smashing d/transact as fast as the user types iirc – not sure if i read it properly – this is probably a bad fit for dom/on (which we want to replace anyway due to all these surprising semantics)#2023-12-2219:54Dustin Getzthe dom/on! pattern bypasses all the surprising machinery, we like this primitive better#2023-12-2220:34Dustin GetzI may be mistaken, reading the src I see that dom/on uses the strategy for each subsequent event to supersede the previous event, canceling and discarding the prior result, so that should be fine for smashing d/transact#2023-12-2820:06Dustin Getzupon re-reading this, i'm not sure that the behavior @U050CJFRU reports ("something refreshing in the UI cancels the flow") matches the intended behavior of dom/on, I think there still may be a userland mistake that is fixable. As far as we are aware, dom/on does the right thing OOTB for this use case of streaming rapid data entry to the server. @U050CJFRU if you want to dig deeper, i'd request a minimal repro in the form of a single file in a fork of the https://github.com/hyperfiddle/electric-fiddle, with no added dependencies (no datahike), just println on the server. I believe we will find that the println is called in the intended way without "data loss"#2023-12-2820:11denikthanks @U09K620SG you might be right. I tried boiling it down to a minimal repro without extra deps and couldn’t find the issue yet. Planning to rewrite that part of the code which might help resolving the issue. I also wonder if this has to do with the way a datalevin database is watched. a while ago Leo suggesting using this https://github.com/lumberdev/tesserae/blob/master/src/tesserae/ui/electric_util.cljc#L115-L127 impl rather than e/watch. Does that still apply?#2023-12-2820:28Dustin Getzi don’t understand what that is working around #2023-12-2820:29denikSame here. I remember the UI would lock up sometimes without it#2023-12-2820:31Dustin Getzdo you have the original thread with leo?#2023-12-2820:31denikhttps://clojurians.slack.com/archives/CL85MBPEF/p1681908108283909?thread_ts=1681842294.096279&cid=CL85MBPEF#2023-12-2820:35Dustin Getzok, I don't see how this could be related#2023-12-2820:39Dustin Getzoh, actually maybe it can be related#2023-12-2820:41Dustin GetzOk i dont know if this is going to impact your issue, but I detect a minor correctness issue in the transact snippet from the other thread#2023-12-2820:42Dustin Getzi want you to wrap all calls to db/transact! in e/offload, like https://github.com/hyperfiddle/electric-xtdb-starter/commit/99d9aac82aba3c7d14a81ee928ef031ea776a22b#2023-12-2820:43Dustin Getzpresumably that call can block for some period of time, and you are hitting it rapidly, this could cause a backup of blocking calls, which will block the electric server process#2023-12-2820:45Dustin GetzI wouldn't expect the missing e/offload to impact your issue (just slow down your app because each transact will block any queries from proceeding in parallel), but the datalevin async-watch locking workaround might have some deep interaction with this that i dont undertand#2023-12-2820:46Dustin Getzalso the report of "data loss" is a hint, i don't have an explanation for that yet#2023-12-2820:47Dustin Getza bunch of backed up blocking calls will be racing against new messages from network that queues up more blocking calls and we potentially desire to cancel stale calls, that is a complex interaction#2023-12-2820:48Dustin Getzthe e/offload, by moving the d/transact! to a thread, makes it cancellable in the first place – without the m/via, i don't think the effect can be cancelled at all it just blocks until it suceeds#2023-12-2820:49Dustin Getzmaybe somehow the wrong one is winning, i'd have to think a lot harder, but the point is – i see a potential interaction#2023-12-2823:52denikThe part that is cancelled is not the transact but a missionary task eval/eval-cell-task that already runs using m/via internally
https://github.com/lumberdev/tesserae/blob/master/src/tesserae/ui/sheet.cljc#L78-L96#2023-12-2307:02braai engineerIs Hyperfiddle UI with HFQL open to use? Gotta build my backyard mechanic a job card system and needs to be fast & cheap to slap some UI on a Datomic schema.#2023-12-2701:00Vincentthinking "yeah"#2023-12-2601:00joshchoi want to tag electric (like as an account) in my twitter posts but can’t, would love it if this becomes possible at a later date#2023-12-2715:46Dustin GetzI've created https://twitter.com/ElectricClojure for the purpose of being tagged in your tweets about Electric, it has links to the github and tutorial etc. I'll improve it over time. Please tag this account in your posts! You can still tag me if you want, the purpose of the new account is to be a "landing page" for newcomers#2023-12-2716:46joshcho:saluting_face: :saluting_face: #2023-12-2722:24joshchowhat’s the electric logo?#2023-12-2722:49joshchoalso, idk if it’s ur style @U09K620SG but @htmx_org on twitter is a v popular software account. worth a reference#2023-12-2715:46Dustin GetzI've created https://twitter.com/ElectricClojure for the purpose of being tagged in your tweets about Electric, it has links to the github and tutorial etc. I'll improve it over time. Please tag this account in your posts! You can still tag me if you want, the purpose of the new account is to be a "landing page" for newcomers#2023-12-2811:55Hüseyin YavaşHi, I have a question about storing login credentials. https://stackoverflow.com/questions/77711194/best-practice-for-storing-login-credentials-in-clojurescript is stack overflow link. Do you have any suggestions?#2023-12-2817:23Vincentwhy is the only answer so far 'don't store them"? 😆#2023-12-2817:24Vincentclearly you need to store them on the clientside, you need to not leak client info but keep it in localstorage#2023-12-2817:25Vincentwhat can you keep that won't let others mess with an account? you can "sign every request" and check the signing, you can use the JWT approach and send a "header, payload, checksum" where the header has an algorithm for writing the checksum specified. then you only accept the payload if the checksum matches what you think it should -- because then you know the data wasn't tampered with#2023-12-2817:25VincentPeople are so averse to answering simple questions with confident answers, I don't get it, someone who actually has this problem should answer 😂#2023-12-2817:35Dustin GetzI think the real question is, "How do have a persistent session in Electric" (as stated in the top comment). The serious production answer is that there are many different ways you might want to do this with varying implications, and most require backend integration at the http server level. The quick and dirty answer is I think what Vincent said – you can use localstorage#2023-12-2817:38Dustin GetzWe're upgrading to Ring 1.11 soon which just got websocket support; this will allow us to drop the jetty boilerplate and do everything using the typical Clojure approaches#2023-12-2817:39Dustin GetzAfter we do this, we'd like to add a better auth strategy to the starter app (perhaps even with Auth0 or something like that so that you get single sign on out of the box). No promises for when we do this#2023-12-2817:40Dustin GetzUsers are welcome to contribute this, none of this requires any Electric internals knowledge#2023-12-2913:38HendrikI store a session ID in local storage and then check that id on the server. Roughly:
(e/client
(let [id (read-from-local-storage)]
(e/server (when (is-authenticated id))
(do-something))))#2023-12-2915:46danbuneaabout auth0 or some SSO I'd be willing to help, if you need me#2023-12-2917:01Dustin GetzPR's or even demo repos welcome, if you decide to work on this I'd be happy to do a zoom call#2023-12-3011:50danbuneaI am not very experienced with auth0, but I will make an attempt at the beginning of January to make it work.#2023-12-3013:29Dustin Getzok let’s definitely zoom i have an old auth0 integration lying around from hyperfiddle-2020 that at least has all the java and jwt code bits you’ll need#2023-12-3015:38danbuneaOk, but it’ll have to be after Jan 2nd #2023-12-2900:01joshchoif i want to create a blog w/ interactive essays, what’s the best way/good references?#2023-12-2900:02joshchois the main website of demos on github? would love to see how that’s done#2023-12-2900:17Dustin Getzthose will be migrated to the electric fiddle repo first week of jan, are you able to wait?#2023-12-2900:19Dustin Getzhttps://github.com/hyperfiddle/electric-examples-app is the repo but electric-fiddle is about to supersede it, this whole markdown authoring thing is the basis for that#2023-12-2900:19joshchoah gotcha#2023-12-2900:20joshchoyeah any existing examples are great, trying to implement before end of yr#2023-12-2922:39Vincentpardon me if this is obvious / in the examples already but is there an upload file example?#2023-12-3001:13Dustin Getzthere’s not but there was some discussion 9? months ago#2023-12-3022:40Vincenthttps://clojurians.slack.com/archives/C7Q9GSHFV/p1676839183601489?thread_ts=1676734391.918809&cid=C7Q9GSHFV
Found it.
Although my implementation does not work yet ;D#2024-12-3114:48VincentGot file uploads working 😄 using @carsten.behring’s work#2024-12-3114:48VincentGot file uploads working 😄 using @carsten.behring’s work#2024-01-0105:05joshchohappy new years everyone 🎉 #2024-01-0105:07Olav FosseHappy new year Josh!#2024-01-0111:36grounded_sageUnsure what happened but the IC version of electric has broken nearly all my PixiJS and D3 code. Things sometimes render and sometimes don't.#2024-01-0111:39grounded_sageI don't get errors. Just values seem to become non-existent and functions not called. I'm doing many printlns but the behaviour of sometimes rendering and sometimes not is throwing me.#2024-01-0111:44grounded_sageMy only theory is that IC doesn't like this kind of code.
(e/def datetime-scale
(e/client
(let [datetime-range (or dynamic-datetime-range static-datetime-range fixed-datetime-range)]
(->
(d3-scale/scaleTime)
(.domain (clj->js datetime-range))
(.range (clj->js [0 panel-w]))))))#2024-01-0112:02grounded_sage(dom/div
(dom/p (dom/text (str (.ticks price-scale))))
(dom/p (dom/text (str (.ticks datetime-scale)))))
Only one of these will render at a time. If I comment out one the other renders. If I have both of them active only the first will render.#2024-01-0112:06grounded_sageI also need to have these values rendering to the dom for the values to be passed to my PixisJS draw function. Otherwise the values are not there. Even my println's inside the let bindings of an e/client before the function call do not print out the values. It only properly renders the values when it is written to the dom.#2024-01-0112:37Dustin Getzi suspect a desync, send us the repo#2024-01-0112:38Dustin Getzlast time this happened to you, there were websocket configuration shenanigans in your repo (local dev build hacked to connect to prod hosted backend), does the repo have anything like that?#2024-01-0112:40grounded_sageNope. I'm avoiding hacks as much as possible.#2024-01-0112:43grounded_sageI'll see if I can get together a minimum repo so there is less going on.#2024-01-0318:08grounded_sageWhen I do a watch on a serverside atom on IC branch I get a number instead of the value.
(defonce !positions (atom "hello"))
(e/def positions (e/server (e/watch !positions)))
Result is 1700659560000
Could this possibly be the same issue? Wondering if you may need the whole repo and not just a slimmed down one.#2024-01-0318:09grounded_sageMeanwhile this works in the same namespace.
(e/def live-quotes (e/server (e/watch polygon/!live-value-quotes)))#2024-01-0318:52Dustin Getzyes that looks like a desync bug, which is possible under IC which is still very new#2024-01-0323:14grounded_sageI just saw the fiddle repo. Mine is a merge of IC branch against the OG electric-starter and there could be some errors or missing updates on my side from that. I'll see about a migration to the fiddle and it may resolve my issues#2024-01-0517:33grounded_sageI am cleaning this all up and actually going to share the repo with what I have instead of porting to fiddle repo since that doesn't have IC yet.
One thing I noticed is that when doing (e/def live-quotes ...) I can't arbitrarily use that in frontend of backend. When I reference live-quotes in a component it needs to be wrapped in (e/server ..) . Is this the expected behaviour?
I will also put together a list of what I find with a supporting video if that is also necessary to aid in description.#2024-01-0517:34Dustin Getzelectric-fiddle repo does have IC#2024-01-0517:35grounded_sageAh okay. I thought it didn’t. Well I will also port across to that for consistency. #2024-01-0121:43Vincent(edit: nvm)
So I have this problem where I think I'm smart sometimes...
(e/defn TestArea []
(e/client
(ui/button
(e/fn []
(e/server (lets-make-a-test-product)))
(dom/props {:class "xt1"})
(dom/text "Fashion a test product"))))
and I define lets-make-a-test-product as a fn
#?(:clj (defn lets-make-a-test-product [] ...))
But I get Unable to resolve symbol: lets-make-a-test-product
Why could this be?#2024-01-0121:45VincentBecause no e/offload?#2024-01-0122:30Vincentnvm i think it was something else#2024-01-0200:15VincentThe problem was that I forgot to include clojure.string :as str in the requirements and it was breaking on not knowing str but would report the earliest invoking component as the culprit. just a head's up in case you see something weird, start removing layer by layer until you find the actual exception.#2024-01-0200:37VincentAny way for me to adjust the default ports so I can run 2 electrics on the same box? Default is 8080 ... with websocket redirect being seamless... 2 instances i might have to adjust the web socket connect port?#2024-01-0202:06Vincentwill let you know how it goes 😅#2024-01-0202:06Vincentdon't feel like provisioning another multi gb ram box x)#2024-01-0204:09VincentI found this page https://github.com/ring-clojure/ring/wiki/WebSockets
but still not sure how to have 2 apps use different websocket ports. since they share the same IP i think they need different ports, unless somehow i can fool the resolution mechanism with just the domains being different? :thinking_face:#2024-01-0204:21VincentOh yay got it figured out. in your nginx config you need this (forward <wss://yourdomain.co> ) on the "/" (root) location...
location / {
proxy_pass ;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
if you are using port 8181 for your application, or Electric application #2, for example.#2024-01-0204:21Vincent(notice the upgrade headers)#2024-01-0215:49JHi guys! I start to update a project with the IC version but I got this error when I eval my dev namespace.
clojure.lang.Compiler$CompilerException: Syntax error macroexpanding at (hyperfiddle/electric.cljc:203:1).
#:clojure.error{:phase :execution, :line 203, :column 1, :source "hyperfiddle/electric.cljc"}
....
....
Caused by: java.lang.ClassCastException: class clojure.lang.Var$Unbound cannot be cast to class clojure.lang.IObj (clojure.lang.Var$Unbound and clojure.lang.IObj are in unnamed module of loader 'app')
I have update my code for IC like add entrypoint on the server, use boot-client , boot-server and more coloring. What I miss?#2024-01-0218:58nivekuillooks like something is trying to use an unbound var but I think this is too vague to be helpable. line 203 is a comment on master so maybe your version is old?#2024-01-0300:55Dustin Getzcheck definition order, electric used to be more relaxed about this#2024-01-0300:55Dustin Getzalso reread the change log just to be sure you understand all breaking changes#2024-01-0300:56Dustin Getzi think you can comment-bisect this, and let us know what it was in the end and i’ll make a note to improve the error message#2024-01-0310:03JOk so I have something! And for easily reproduce it, I'm based on the electric-starter-repo. My main project use rama (https://github.com/jeans11/demo-rama-electric) on version 0.10.0 and electric v2-alpha-428-g22937f75.
If I update only rama to the 0.11.4 release. No problem.
But if I update electric to the IC version (commit d5b56355628b32326477072f83510f5b657b4211 on master). I got the error.
If I remove the rama deps, the error gone.
So there is something between the IC version and rama.
Here the whole stacktrace => https://gist.github.com/jeans11/f27d4ccab6448ae03b1dea066e050580#2024-01-0312:20Dustin GetzWhich var is unbound? The error came from somewhere#2024-01-0315:44JBetter error message:#2024-01-0315:47JTo reproduce:
• checkout electric-starter-app on the ic branch
• update electric to the last commit on master (`d5b56355628b32326477072f83510f5b657b4211`)
• add com.rpl/rama {:mvn/version "0.11.4"} in :deps
• launch clj -A:dev -X user/main#2024-01-0421:03JI don't understand this error on partial-dynamic . I see the macro in hyperfiddle.electric 😕#2024-01-0421:10xificurCI think the stacktrace in the gist is the one with more helpful info, the "caused by" part throws https://github.com/clojure/clojurescript/blob/22e3dc42d60747c3187bedb82079101caf049549/src/main/clojure/cljs/analyzer.cljc#L664. with-meta is called on an unbound var bound to a data reader#2024-01-0510:37JWhat is the goal of the ?enrich-for-require-macros-lookup function in hyperfiddle.electric.impl.expand?#2024-01-0513:04JFound it!#2024-01-0517:20JSo for the moment the workaround to use rama and the IC version of electric is to require org.clojure/data.xml {:mvn/version "0.2.0-alpha8"}. The old version of electric works perfectly with rama.#2024-01-1105:06weithanks for sharing, i'm also using rama and this workaround fixed the issue for me.#2024-01-0218:56nivekuilI see that timbre was dropped in favor of a simpler approach in electric, is this also what hyperfiddle does in applications or is timbre still used there? I see timbre's value in compile time elision and a cross platform API#2024-01-0219:01Dustin Getziiuc (and I may be misremembering), electric core doesn't log anymore, all namespaces that log are in electric-fiddle (i.e. jetty, electric jetty adapter, datomic, shadow)#2024-01-0219:02Dustin Getzjetty logs using standard java log interfaces, so if you are using our jetty boilerplate — which is optional, strictly speaking — you are forced into SLF4J or whatever#2024-01-0219:03Dustin Getztimbre is a pure clojure solution that doesn't use any of the standards#2024-01-0219:03Dustin Getzyou are welcome to use it in your applications#2024-01-0220:12nivekuildo you use timbre? not sure if there's anything else out there for cljc#2024-01-0220:13nivekuiland electric.cljc does log, looking at master#2024-01-0220:15Dustin Getzok, i see you're right that we still log#2024-01-0220:16Dustin Getzwe were unhappy with timbre last time we used it for anything, i've forgotten the details#2024-01-0220:16nivekuilyeah, I've noticed some hostility against it especially for libraries. but I don't see anything better#2024-01-0220:16Dustin Getzi personally like (e/server (log ...))#2024-01-0220:18Dustin Getzregarding the logging in electric, i think we will try to move that into electric-fiddle so that the core library doesnt force the dependency. the electric-fiddle repo we intend to become important#2024-01-0220:20Dustin Getzelectric-fiddle is becoming a batteries-included application server, maybe even with a CLI so you can "hyperfiddle serve foo.cljc" with free routing, auth, logging etc – just write top-level e/fns#2024-01-0220:25nivekuilthat's cool, I wanted something to do something similar + a tailscale sidecar for public dns/ACLs, sounds like I should be building on top of fiddle#2024-01-0220:26nivekuilnot my priority to be clear, that would probably be better profiling/visibility (less continuation soup, more granular stacktraces than e/defn) and probably differential#2024-01-0315:53andrea.crottiAm I the only one getting this sort of errors?
I was just trying to run locally either:
https://github.com/hyperfiddle/electric-starter-app
or https://github.com/hyperfiddle/electric-xtdb-starter
and tried with jdk 11/17/21 but I always get this error
Caused by: java.lang.NoClassDefFoundError: com/google/common/collect/Streams
at com.google.javascript.jscomp.deps.ModuleLoader.createRootPaths(ModuleLoader.java:259)
at com.google.javascript.jscomp.deps.ModuleLoader.<init>(ModuleLoader.java:154)
at com.google.javascript.jscomp.deps.ModuleLoader$Builder.build(ModuleLoader.java:146)
at com.google.javascript.jscomp.deps.ModuleLoader.<clinit>(ModuleLoader.java:414)
at com.google.javascript.jscomp.DiagnosticGroups.<clinit>(DiagnosticGroups.java:178)
at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.ensureClassInitialized(MethodHandleAccessorFactory.java:300)
at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.newMethodAccessor(MethodHandleAccessorFactory.java:71)
at java.base/jdk.internal.reflect.ReflectionFactory.newMethodAccessor(ReflectionFactory.java:159)
at java.base/java.lang.reflect.Method.acquireMethodAccessor(Method.java:726)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1726)
... 211 more
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Streams
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
... 225 more
#2024-01-0315:54andrea.crottihehe dammit it was one of the libraries I had in my ~/.clojure/deps.edn causing some sort of conflict#2024-01-0318:25Vincentoh hell yeah#2024-01-0318:25Vincentglad you got it resolved#2024-01-0318:25Vincentit's always nice when it's simple stuff like that#2024-01-0318:56Dustin Getzthis is a common known ClojureScript issue, you'll get a number of hits if you search the broader slack (some in this channel)#2024-01-0318:26VincentMoving very fast with Electric, a three day memoir...#2024-01-0318:43VincentI thought I would use Rum to develop Shomp -- our application to help musicians and creators have perpetually free online storefronts -- and i thought to use Rum because we can have serverside rendering out of the box via (rum/render-html) and (rum/hydrate).
However, after smashing my head against a wall for days, working on connecting front end and back end APIs in a "beautiful, reusable 'datomic push' syntax" way, I found myself reaching for Electric patterns in a non-electric land.
So i bit the bullet and decided to start writing the application in Electric instead. Not disappointed, developing the app with Electric has proven to be: more fun, and simpler to reason about.
Late last night I just got the one time passcodes via text message sending. Apparently small things like "putting a url in a text message" will invisibly filter the message on the network and never deliver -- you can imagine the tech support email chain around mysterious undefined behavior lol.
I decided that we can still implement an inexpensive and robust SEO solution that google can use when learning about new pages, and real users can hit the Electric application directly. I think that as long as I can provide meta-description and those meta tags for the SEO bots it will be quite alright on the search engine side.
Anyway, gotta say, it's really hard to go back to "last generation tooling" once you've made a couple projects hum along smoothly in Electric lady land.
It really is the perfect (approaching perfection) medley of capabilities to create, in exceptionally rapid fashion, a proof of concept that is easily massaged into a minimum viable demo and subsequently minimum viable product. Coding live and in production like a cowboy. Yee haw.
There are many ways to view what is happening, in one way of looking at it, Electric Clojure lets you "bolt on" the amenities and functionality of the server directly into the client/clientside/the browser. This transparent reference to the methods and collections (data) on the server is invaluable.
Another way is to consider that you can make a multiplayer-enabled app out of the box, which might be a great fit for some use-cases.#2024-01-0321:29Dustin GetzThanks for the review, I especially appreciate that you tried to do it the other way! Electric is not perfect yet and it's hard to get a fair comparison#2024-01-0322:51markaddlemanI think I found a typo in the electric-fiddle readme. I believe
(dev/load-fiddle! 'my-fiddle)
should be
(dev/load-fiddle! 'my-fiddles)
(note the “s”)#2024-01-0413:31Geoffrey GaillardFixed. Thank you for the report.
You are right, 'my-fiddle was sometimes singular and sometimes plural. Now it’s all singular.#2024-01-0419:01shaneAre there any examples/is it possible to use electric with pedestal right now?#2024-01-0419:22Dustin Getzno#2024-01-0419:23Dustin Getzthe pure ring stuff just wrapped up internally - release imminent- not sure if related. this changeset allows users to kill the jetty boilerplate for the ws#2024-01-0518:08bhurlow^ great, excited for that#2024-01-0512:30Stephan@dustingetz FYI https://twitter.com/dan_abramov2/status/1743241094412820701#2024-01-0513:09Dustin GetzDan needs to try harder#2024-01-0518:07bhurlowopens link, sees lisp, puts laptop in industrial incinerator#2024-01-0518:57Vincentwhat is RSC?#2024-01-0518:57Vincentnew version of star craft i hope#2024-01-0521:39elkenReact Server Components I assume#2024-01-0521:45VincentOh nice. Yeah that sounds very relevant then. Uh I think the comparison is something like railway to fibre optic?#2024-01-0523:37weiWe can at least like the tweet mentioning electric though!#2024-01-0521:03VincentIn my latest in-roads on the adventure to smooth authentication of electric applications I have a solution in mind and I would like your feedback.#2024-01-0521:07VincentA picture worth at least 2^10 words...#2024-01-0521:11VincentThe only time we can read headers of the page is on page load, effectively insulating your electric application from the ring middleware stack.
In my case, I want to collect a phone number, send a 2FA code, and confirm it to be true, I can do this in electric lady land in nested (e/client (e/server..)) blocks
I can also save directly to the [shared] XTDB instance which you can see as the diamond between the "moon and the sun" or whatever we can wall/call them.
So, save to the database an "active session"
{phone {:expiry 123456475
:session-val "SEUTHOS751642753"}}
Okay, index by phone number, generate a session value in (e/server) save to xtdb
Then, send an "over the air, over the wire" http POST to your Ring Middleware stack for your app. This POST shall: contain the phone and the session value, which the middleware will check against the xtdb values, confirm within expiry by whatever buffer, either mint a fresh session if close and otherwise, redraw the ring map = update the headers and finally, trigger a page refresh on the client via the successful response from that little blip across the waters there (an HTTP post)#2024-01-0522:45VincentI know the boys are gonna throw tomatoes at me but...
it's not an Electric client
or server
it's a hybrid
clerver?
slient?
okay maybe wordsmash not gonna make the grade...#2024-01-0523:56grounded_sageI dno seemed pretty clerver to me#2024-01-0604:04joshchoI am trying to change the order of e/defn via declare, but I know that declare isn't a thing yet. What is the best workaround for forward declarations?#2024-01-0612:46Dustin Getzprobably letfn#2024-01-0606:23nivekuilI noticed the de branch added a triple store for the compiler. what's new about differential that warrants this change? any further plans for integrating a triple store into electric?#2024-01-0617:12xificurCIt's an implementation detail and a personal choice. Subject to change. The reason is mainly data flexibility#2024-01-0609:22joshchorunning ic branch rn, and ran into some issues trying to deploy with java -DHYPERFIDDLE_ELECTRIC_SERVER_VERSION=$HYPERFIDDLE_ELECTRIC_APP_VERSION -jar app.jar :
Exception in thread "main" clojure.lang.ArityException: Wrong number of args (1) passed to: app.electric-server-java8-jetty9/start-server!
at clojure.lang.AFn.throwArity(AFn.java:429)
at clojure.lang.AFn.invoke(AFn.java:32)
at prod$_main.invokeStatic(prod.clj:13)
at prod$_main.doInvoke(prod.clj:10)
at clojure.lang.RestFn.invoke(RestFn.java:400)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:135)
at prod.main(Unknown Source)
#2024-01-0612:43Dustin Getzfirst, use master, the branch has merged and there may be more commits#2024-01-0612:43Dustin Getzcheck the changelog, there was a breaking change related to env vars#2024-01-0612:44Dustin Getzyou can compare to the dockerfile in electric fiddle repo, as well as the starter app commit linked in the readme (i think the starter app changes are not in starter app master because we havent created a maven release yet)#2024-01-0706:38joshchogotcha, i’ll try that#2024-01-0708:52mattiasIn https://github.com/hyperfiddle/electric-xtdb-starter/blob/master/src/user.clj we define user/!xtdb var, which works well for dev setups. In https://github.com/hyperfiddle/electric-starter-app/blob/main/src/prod.clj, you define prod as a separate namespace that just starts jetty. How would you handle a case when in prod we have a need to define such a stateful !xtdb var or e/def?
In my project I have several such stateful objects/vars (for xtdb, graalvm context, playwright browser) which I initialise in user.clj and then later do (binding [!xtdb @(requiring-resolve 'user/!xtdb)]) . How should I approach this in uberjar deployments? Should I create some shared-vars namespace and (alter-var-root #'shared-vars/!xtdb) from both user.clj and prod.clj? Or alter-var-root does not work well with uberjars?#2024-01-0721:06JAtkinsIs there an easy way to boot electric in a server-only mode?#2024-01-0800:57Geoffrey GaillardWe have an exemple here. No official support. local-def was initially built for testing only.
https://github.com/hyperfiddle/electric-fiddle/blob/0c11afadf1374666e502c700be18360fe57f3998/src-dev/fiddle_manager.clj#L168#2024-01-0800:58JAtkinsSweet! Yeah I’m interested in a POC - nothing solid is required :). Thanks!#2024-01-0723:40VincentI have re-thought my strategy for user phone number logins. Rather than write session data to the database, you can simply have the tuple/quadruple
{:phone 777777777 :session "x2345" :expiry 99999999 :signature "signature-made-by-applying-algorithm-to-phone,sesh,expiry-with-your-secret-key-for-making-sigs"}
and then the middleware can simply double check that the signature is valid for the given :phone, sesh, expiry
ta.da.#2024-01-0800:21Vincentone drawback being that manually expiring tokens will require some statefulness and that requires a db.#2024-01-0805:36Vincentokay my latest and greatest strategy:
do OTP validation in electric app
set cookies in electric app via (set! (.-cookie js/document) (str name "=" value";"))
immediately after, hit the server endpoint which confirms the sig is correct and if not, clears cookies. if good, returns cookies as they are. seems to be effective. time to shower xD#2024-01-0823:17JAtkinsI've searched through the chat a bit but haven't seen anything relevant - is there a nice way to bridge electric-native code and clojure written missionary code? UI4 seems to use e/listen>, creating a flow that ultimately directly invokes an e/fn. How would I go the other way and make a electric value a missionary stream?#2024-01-0823:20Dustin Getz(e/fn [] x) type checks as a missionary flow of each successive value of x#2024-01-0823:21Dustin Getzmaster has e/flow i think, not sure if that got published yet#2024-01-0823:21JAtkinssounds good. I'm screwing around with HumbleUI, and I can't use electric everywhere (humble is very protocol heavy), so the context switch is quite useful.#2024-01-0912:01joshchois there a way to delay the state of a watch value? something like (delay some-watched-value 1000), which would receive 1s delayed input from some-watched-value#2024-01-0912:21xificurChttps://github.com/hyperfiddle/electric/blob/d5b56355628b32326477072f83510f5b657b4211/src/contrib/electric_contrib.cljc#L6#2024-01-0912:50joshchoty @U09FL65DK !#2024-01-0916:10joshchothere is a bit of an issue with trying to use delay and reset. i am trying to reset a server atom within the reduce and having some issues:
(let [!cancel (atom nil)]
(letfn [(delayed-seed [d xs]
;; see missionary slack
(m/ap (m/? (m/sleep d (m/?> (m/seed xs))))))]
(when @!cancel
(@!cancel))
(let [saved @!draft-relay
cancel
((->> (subs latest-msg (count saved))
(delayed-seed 140)
(m/reductions str)
(m/reduce (fn [_ partial-msg]
;; PROBLEMATIC
(e/server (reset! !draft-relay
(str
saved
partial-msg))))
nil))
#(do (js/console.log :success %))
#(do (js/console.log :failure %)))]
(reset! !cancel cancel))))
any solutions/workarounds would be appreciated.#2024-01-0916:10Dustin Getze/server not allowed in clojure lambdas#2024-01-0916:11joshchoi understand it's not, but how would i solve this problem?#2024-01-0916:13joshchoessentially i am receiving large partial strings (e.g. "abcd", "abcdefg", "abcdefghij"), and i want to stream these as "a", "ab", "abc", "abcd", "abcde", "abcdef", etc. as soon as they are available.#2024-01-0916:13Dustin Getzm/reductions returns a flow, join that flow back to electric with new. You'll need to give it an initial value, as delayed-seed does not have initial value#2024-01-0916:14joshchoi figured out a way in clojure missionary, but having a hard time moving this to electric
(->> (m/ap
(let [v (m/?> (m/seed (m/?> (m/seed xs))))]
(m/? (m/sleep 200))
v))
(m/reductions str)
(m/reduce (fn [_ m]
(prn m))
nil)
m/?)#2024-01-0916:14joshchogotcha#2024-01-0916:14joshchoi'll try that#2024-01-0916:15Dustin Getz(when-some [x (new (m/reductions ...))]
(e/server (reset! !x x)))
#2024-01-0916:15Dustin Getzthat is basically a callback, reset! will run when x changes#2024-01-0916:15joshchowill this work also in e/fn setting?#2024-01-0916:15Dustin Getzi dont understand#2024-01-0916:16joshchomy understanding is e/fn runs once and doesn't persist#2024-01-0916:17Dustin Getzi stil don't understand, also what e/fn are you talking about?#2024-01-0916:17joshcho#2024-01-0916:17joshchofor context#2024-01-0916:17Dustin Getzthe lifecycle of the e/fn is determined by the caller#2024-01-0916:18joshchohmm#2024-01-0916:18Dustin Getzhttps://electric.hyperfiddle.net/user.tutorial-lifecycle!%4Cifecycle#2024-01-0916:18Dustin GetzI see you're doing UI code, is this a callback to a hyperfiddle.electric-ui widget or something?#2024-01-0916:24joshchono, just using dom#2024-01-0916:24joshchoi tried the approach you outlined but not quite working unfortunately#2024-01-0916:25joshcho(when-some [x (->> "hoeutnhoetun"
#_(subs latest-msg
(count draft))
(delayed-seed 140)
(m/reductions "" str)
new)]
(js/console.log x)
;; (e/server (reset! !draft))
)#2024-01-0916:26Dustin Getzm/reductions args look flipped#2024-01-0916:26joshchooh woops#2024-01-0916:27joshchoit just prints the empty string once#2024-01-0916:28Dustin Getztry at the top of your app#2024-01-0916:29joshchoyeah top of the app works!#2024-01-0916:29joshchoso should i handle this somewhere other than within the e/fn?#2024-01-0916:29joshchoi guess i still don't understand lifecycles#2024-01-0916:30Dustin Getz"other than within the e/fn?" this doesn't really bind to meaning for me#2024-01-0916:30Dustin Getzare you using dom/on to call the e/fn that you put this code in?#2024-01-0916:30joshchoyes#2024-01-0916:30Dustin Getzok, that is the source of the problem#2024-01-0916:31Dustin Getzhonestly just don't use dom/on, use dom/on! instead, i think there are some threads showing how to migrate#2024-01-0916:31joshchowhat's the difference?#2024-01-0916:31Dustin Getzthe whole idea of a "callback" is incoherent in electric, because callbacks don't have latency#2024-01-0916:32Dustin Getza better pattern is (when-some [e flow-of-events-from-missionary] (println e))#2024-01-0916:32joshchohmm#2024-01-0916:33Dustin Getzthe -some lets you use nil as an initial value#2024-01-0916:33joshchobut could i still update server atoms even in clj functions like that?#2024-01-0916:33Dustin Getzthat's an electric block#2024-01-0916:33joshchooh ok#2024-01-0916:33joshchoare there any examples of this?#2024-01-0916:33Dustin Getzi think in the search#2024-01-0916:33joshchookay i'll do that after sleeping#2024-01-0916:33joshchocan't think rn#2024-01-0916:33joshchothank you#2024-01-0916:45VincentHi I have a component where I have a watch on a clientside value and there's a serverside value...
and I want the ui/button to initiate a series of events only when the clientside value equals the serverside value (one-time passcode via text message)#2024-01-0917:51Dustin Getzcan you use if ?#2024-01-0919:46Vincentyes. it was not working, but now it is. =)#2024-01-0916:56VincentI notice dom/on "change" for input box does receive an event
but only when i focus out / off focus the box. not on each char/keystroke#2024-01-0916:59s-olthat's just how JS events work, you might be looking for the input event instead:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/change_event
> The change event is fired for <input>, <select>, and <textarea> elements when the user modifies the element's value. Unlike the input event, the change event is not necessarily fired for each alteration to an element's value.#2024-01-0917:04Vincentoh, cool!#2024-01-0917:04Vincentthanks 😄 ❤️#2024-01-0917:04Vincent0 hairs pulled out in this learning process, appreciate it.#2024-01-0917:30Vincent(dom/on "keyup"...) also very nais [borat voice]#2024-01-0917:30Vincentkeyup/keydown#2024-01-1003:24Vincentinput is the answer#2024-01-0917:30Vincent(dom/on "keyup"...) also very nais [borat voice]#2024-01-0920:12Tommy JollyboatI've migrated from electric-starter-app to hyperfiddle/electric-fiddle, and Electric's server no longer follows symlinks (I had one in`resources/public/css` that pointed to CSS that I share between projects). I'm going to learn about git submodules and do it that way instead, but I thought I'd make a note here in case this is an unintentional change!#2024-01-0920:32Dustin Getzwe didn’t know, thanks#2024-01-0920:57Tommy Jollyboat[Edit - ignore this stupid question] Is there an elegant way to reload CSS & other resources when the files change? Or alternatively add a "cache buster" parameter to the dom/link or whatever.#2024-01-0921:21Vincenthmm there is some file watching going on. i wonder if css can be added to the list?#2024-01-0921:21Vincentmethinks shadow cljs can watch css :thinking_face:#2024-01-0921:27Tommy JollyboatWell, that's embarrassing. It was working all along. No idea what I did wrong! Thanks!#2024-01-0921:31Vincentif you are interacting with your project over https:// you will lose the hot-reload for css dunno why. atn least that's my experience. so maybe thar?#2024-01-0921:31Vincent"repl repl, your state is a mess" ))#2024-01-0921:32Tommy JollyboatI think it was more like the koan of Knuth power-cycling a lisp machine -- it worked once I understood more about why it should 😛 Thanks for your help!#2024-01-0922:39VincentHaha, it certainly appears that way at times!#2024-01-1013:53joshchohow do i do routing in electric? are there any examples?#2024-01-1013:54Dustin Getzeasiest way is a case statement, if you search for "routing" there are several discussions#2024-01-1013:55Dustin Getzwe have a hyperfiddle.router coming for complex route compositions but it's not ready#2024-01-1013:55Dustin Getzcore.match also works#2024-01-1013:56Dustin Getzthere is a goog.history integration under the contrib namespace#2024-01-1014:32Jcase+`reitit`#2024-01-1016:36danbuneaI learned how to do it from this example: https://github.com/jeans11/demo-rama-electric/blob/main/bases/web-ui/src/dre/web_ui/routes.cljc (reitit frontend based)#2024-01-1017:48CanHi, You can use that template: https://github.com/Bariscanates404/param-routing-electric-datomic-template History router#2024-01-1017:58Rob Haisfield@dustingetz Is Electric uniquely possible in Clojure, or would it work in any language that has macros, like Rust or Swift?#2024-01-1117:28Dustin GetzStrictly speaking, maybe possible. Macros not strictly needed, the compiler can be language-external (i.e. Babel). Immutability-by-default is important to the reactive dataflow semantics. Mutation tracking (for imperative langs such as JS or WASM) is a hard unsolved problem, only beginning to be tackled in the literature i've seen. So technology is not there yet. More loosely speaking, perhaps a 65% good enough solution could get adoption in mainstream, but you'd still need a modern concurrency library and there's nothing good enough in JS that I've seen (we use Missionary which is the state of art, years ahead of effect-ts)#2024-01-1117:30Dustin GetzClojure is the right place to do an experiment like this and be able to move reasonably quickly, and in a commercial setting with PL enthusiasts to early adopt and talk to us while we figure out the design, etc#2024-01-1023:42joshchothis is kinda tough to describe, but i will attempt it anyhow:
there is s1, s2, s3, ... and these are coming in at different times sequentially.
i want to stream (in the ui) s1, s2, and s3 one character at a time, but stream in s2 only after s1, s3 only after s2, etc.
so if s1 = "hello" and s2 = "there", then the user would see h, he, hel, hell, hello, then once s2 is received, hellot, helloth , etc.#2024-01-1023:43joshchothis is what i have currently, which doesn't do any accumulation
(let [!q (atom "htht")
q (e/watch !q)]
(letfn [(random-delayed-seed [fixed-delay variable-delay xs]
(m/ap
(m/?
(let [x (m/?> (m/seed xs))]
;; order matters here
(m/sleep
(+ fixed-delay (rand-int variable-delay))
x)))))]
(when-some [x
(->> q
(random-delayed-seed 50 150)
(m/reductions str "")
new)]
(js/console.log x)
(div (text x))))
(div (text q))
(dom/input
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(reset! !q (.. e -target -value)))))))#2024-01-1023:52Vincentjosh this is really interesting what's your desired outcome? trickle-write of any streamed input in any asynchronous order received?#2024-01-1023:53joshchoyep#2024-01-1023:53joshcho(for llms)#2024-01-1023:55Vincentis there a good reason the llm would give you a strange output sequence ? or is that more of a design consideration.#2024-01-1023:56joshcholmao it's a result of a hack i am doing for our project#2024-01-1023:56joshcholong story#2024-01-1023:56joshchoit's a very strange use case#2024-01-1023:57joshchobut very critical for our use case#2024-01-1023:59joshchoalso a hunch that this is much more easily solved using differential electric#2024-01-1100:03Vincenthow do we know the appropriate sequence of stuff? is everything tagged with a sequence number on the way in?#2024-01-1100:04joshchoyeah s1 s2 s3 are sequential inputs#2024-01-1100:04joshchoreset! of an atom#2024-01-1100:06VincentOh okay, they are coming in in the correct sequence already, gotcha. I reckon putting it on a channel is the right answer#2024-01-1100:07joshchoi haven't dealt with channels too much, how would i do that?#2024-01-1100:07Vincentbasically using put and take to load and unload the chan#2024-01-1100:07joshchohow does that translate to electric#2024-01-1100:08joshcho\ are there examples?#2024-01-1100:08Vincentput and take utilize go blocks, i wonder abt electric examples#2024-01-1100:08joshchoi am assuming just reset! of atom is not enough because it'll completely forget about previous values#2024-01-1100:09Vincentyeah some sort of state tracking#2024-01-1100:09Vincentseems necessary#2024-01-1100:09joshchoyeah#2024-01-1100:10Vincenttold chatGPT4 to generate an example with put and take, does it look right ?
(ns your-namespace
(:require [clojure.core.async :refer [chan put! take! go]]
[cljs-http.client :as http]))
(defn get-llm-response [request]
;; Function to get a response from the LLM (simulated here as an HTTP request)
;; In a real scenario, you would replace this with an actual HTTP request to the LLM API
(http/get ""
{:params {:query request}}))
(defn process-llm-response [response]
;; Function to process the response from the LLM
(println "Received response from LLM:" response))
(defn send-request-to-llm [request channel]
;; Function to send a request to the LLM and put the response into the channel
(go (let [response (<! (get-llm-response request))]
(put! channel response))))
(defn main []
;; Main function to demonstrate put! and take!
(let [channel (chan)] ; Create a new channel
;; Send a request to the LLM
(send-request-to-llm "Hello LLM" channel)
;; Take the response from the channel and process it
(go (let [response (<! channel)]
(process-llm-response response)))))
(main)#2024-01-1100:14joshcholols#2024-01-1100:23joshchocan't quite figure it out...
(let [!q (atom "htonuehto")
q (e/watch !q)
ch (a/chan)]
(div (text ch))
(letfn [(random-delayed-seed [fixed-delay variable-delay xs]
(m/ap
(m/?
(let [x (m/?> (m/seed (m/?> xs)))]
(m/sleep
(+ 20 (rand-int 200))
x)))))]
(when-some [x
(->> (channel-flow ch)
(random-delayed-seed 50 150)
(m/reductions str "")
new)]
(js/console.log x)
(div (text x))))
(div (text q))
(dom/input
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(a/put! ch (.. e -target -value)))))))#2024-01-1100:26Vincenthmm#2024-01-1100:26Vincenti think you're on the right track#2024-01-1100:26Vincentnot sure about new)#2024-01-1100:28joshchoi think channel-flow doesn't work in cljs#2024-01-1100:29joshchothis code
(defn forever [task]
(m/ap (m/? (m/?> (m/seed (repeat task))))))
(defn- take! [chan]
(m/via
m/blk
(let [r (a/<!! chan)]
(if (nil? r)
(throw (ex-info "take cancelled, channel closed" {:cancelled? true}))
r))))
(defn channel-flow [ch-recv]
(forever (take! ch-recv)))#2024-01-1100:30joshchonew converts it to electric value#2024-01-1100:32joshchowhatever it is, i keep getting
#object[Error Error: Unsupported operation.] core.cljs:200:23#2024-01-1100:40Vincenthmmm#2024-01-1100:41joshchoomg i did it#2024-01-1100:41joshchoi haven't used my brain this much since my operating systems class lmaooooo#2024-01-1100:42Vincentyay#2024-01-1100:42Vincentgreat job#2024-01-1100:42joshchoall i was missing was e/fn [] that turned it into a flow#2024-01-1100:43VincentOh very cool)#2024-01-1110:29xificurC(m/seed (m/?> xs)) is just xs#2024-01-1200:58joshcho@U09FL65DK i am noticing difference in behavior between those two#2024-01-1102:33weiis there an example for the hyperfiddle.electric-ring-adapter that just landed?#2024-01-1110:14Geoffrey GaillardYes, see: https://clojurians.slack.com/archives/C7Q9GSHFV/p1704967833668529#2024-01-1105:24joshchoi keep getting this error, on ic for electric-starter-app but on master for not
Exception in thread "main" clojure.lang.ArityException: Wrong number of args (1) passed to: app.electric-server-java8-jetty9/start-server!
at clojure.lang.AFn.throwArity(AFn.java:429)
at clojure.lang.AFn.invoke(AFn.java:32)
at prod$_main.invokeStatic(prod.clj:13)
at prod$_main.doInvoke(prod.clj:10)
at clojure.lang.RestFn.invoke(RestFn.java:400)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:135)
at prod.main(Unknown Source)
#2024-01-1105:50weithey added another arg to that function so check that you're calling it with two args, entrypoint and config#2024-01-1106:11joshchowhat's entrypoint?#2024-01-1106:17joshchotrying to update to latest electric-starter-app#2024-01-1110:19Geoffrey GaillardThe entrypoint is the function booting your Electric program. It looks like this:
(fn [ring-request]
(e/boot-server {} your-ns/Your-e-defn ring-request))
See https://github.com/hyperfiddle/electric-fiddle/blob/cf6134adb6aa0a4ae2f678961e54036f867cd4d4/src/electric_fiddle/server_httpkit.clj#L14-L39 and https://github.com/hyperfiddle/electric-fiddle/blob/cf6134adb6aa0a4ae2f678961e54036f867cd4d4/src-dev/dev.cljc#L42#2024-01-1110:21Geoffrey GaillardAlso see the https://clojurians.slack.com/archives/C7Q9GSHFV/p1704967833668529 for simpler entrypoints and simpler ring integration.#2024-01-1105:50weithey added another arg to that function so check that you're calling it with two args, entrypoint and config#2024-01-1106:00PanelCan I embed an electric app inside a react / cljs spa ?#2024-01-1110:25xificurCyes, you can mount electric on any dom node#2024-01-1110:10Geoffrey GaillardNew Electric mvn release {:mvn/version "v2-alpha-540-ga4699532"}.
This is a breaking release.
• Simpler websocket integration – https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#v2-alpha-540-ga4699532--2024-jan-10
◦ Leverages Ring 1.11+ websocket support for simpler ring integration.
◦ Examples for https://github.com/hyperfiddle/electric-fiddle/blob/cf6134adb6aa0a4ae2f678961e54036f867cd4d4/src/electric_fiddle/server_jetty.clj#L15-L33 and https://github.com/hyperfiddle/electric-fiddle/blob/cf6134adb6aa0a4ae2f678961e54036f867cd4d4/src/electric_fiddle/server_httpkit.clj#L14-L39
• Incremental Compilation is included too – https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#v2-alpha-536-g0c582f78--2024-jan-10
◦ TL;DR: Electric used to compile the whole program at once, resulting in long recompilation times in dev mode. Your hot code reload should now be snappy again.#2024-01-1110:12Geoffrey Gaillard@U6D1FU8F2 This will answer your question from the above thread. Electric-starter-app is now deprecated in favor of https://github.com/hyperfiddle/electric-fiddle/#2024-01-1110:14Geoffrey Gaillard@U0GE6JTKK this is the ring release you’ve been waiting for#2024-01-1110:45henrik@U2DART3HA I don’t think the initial IC release was announced in #C06MAR553, so maybe it’s a good opportunity for some promotion of the great work you do?#2024-01-1111:15Geoffrey GaillardThank you simple_smile
But let's wait for feedback a bit, and plan broader announcements as a team first ;)#2024-01-1111:16joshchothanks @U2DART3HA! what is the difference?#2024-01-1112:38Geoffrey GaillardThe difference between electric-starter-app and electric-fiddle?#2024-01-1112:42danbuneaThank you @U2DART3HA 🙂#2024-01-1118:41VincentI will drop this into my deps.edn later on and let you know if I encounter any issues#2024-01-1118:53deniknew httpkit adapter works for me!#2024-01-1200:24joshcho@U2DART3HA yes! trying to determine if to migrate#2024-01-1209:36Geoffrey GaillardElectric Fiddle is the new, easier way to get started with Electric Clojure. We won’t update electric-starter-app anymore. It is worth migrating if
• you plan to integrate updates from the our upstream repo
• your project has not diverged that much from the starter app
Migrating should not take long:
• move your source code to src-fiddles/your-project
• copy the structure from src-fiddles/hello-fiddle
◦ especially fiddles.cljc
• add your extra deps under an alias in deps.edn
• move your resources in resources
• try to run in dev mode (see readme)
• try to run in prod mode (see readme)#2024-01-1307:53Geoffrey GaillardThank you for your feedback.
User code go to src-fiddle, src contains electric-fiddle's source code.#2024-01-1313:28joshchothat feels not ergonomic imho. users will want to use src directory#2024-01-1313:29joshchofor every other project i jump into src, but if it's different for electric fiddle it'll be highly confusing#2024-01-2315:50Geoffrey GaillardWe acted on your feedback. src is now the default place to put user code into. Thank you 👍#2024-01-2409:29grounded_sageI noticed a switch also to using npm install over yarn tbh I haven't really had much of a strong preference and started using yarn because thats what everyone else was doing. Is there a reason for the switch?#2024-01-2409:36grounded_sageAlso is the submodule folder / vendoring now not relevant and you are using deps to manage dependencies?#2024-01-2410:57Geoffrey GaillardThe only reason we used yarn over npm install was because it’s faster. There was confusion over what yarn means, but there was no confusion over what npm install means.
You can use either. Up to you.
> is the submodule folder / vendoring now not relevant and you are using deps to manage dependencies?
yes#2024-01-1210:48VincentIs there a way to conditionally evaluate an (e/server ..) block .. like if and only when the phone number input is 10 chars? otherwise don't evaluate that branch at all, not even a little bit, just assign this other value like "" until the phone input is ten chars long#2024-01-1210:53henrik(e/client
(if (= 10 (count phone-number))
(e/server …)
""))#2024-01-1310:23danbuneaHey, as discussed with @dustingetz I am trying to make an electric demo of authentification using auth0. Something such as: https://www.youtube.com/watch?v=sviForNKVC4 . The code isn't done, and it isn't (yet) adapted to the latest electric-fiddle. It is here: https://gitlab.com/danbunea/electric-ring-auth0-authentification#2024-01-1310:23danbuneaCode repo: https://gitlab.com/danbunea/electric-ring-auth0-authentification#2024-01-1310:26danbuneaIt is pretty simple. Once auth0 authentifies you redirects to your website with a code. You then use that code to make a http call back to auth0 to obtain the logged user info: https://gitlab.com/danbunea/electric-ring-auth0-authentification/-/blame/main/components/auth/src/auth/use_cases/token_from_code/auth0.clj?ref_type=heads#L89#2024-01-1310:26danbuneaI will continue to improve it and to adapt it to electric-fiddle#2024-01-1310:29danbuneaalso the redirect from auth contains a state which you provide when redirecting to it. I'll need to use it to check that I actually triggered the authentification in the first place.#2024-01-1310:29danbuneaThe middleware so far it;s not quite good, but I'll improve it#2024-01-1310:32weithanks for making this! i'm working on an app that will require auth0 so this is perfect timing. will take a look.#2024-01-1310:38danbuneaNot quite done but the main functionality of redirecting and obtaining the user from the code work well#2024-01-1410:54danbuneaYes @U06B8J0AJ sure, I'd like to see it if possible#2024-01-1509:23henrikHere it is : https://gist.github.com/eneroth/1ed58a31f4dc3609309cbb8ff6bfe96e
Some notes:
• It prefers the Auth0 Java API over the JS one.
• The return value is nil while not logged in or logging in, or a map of claims while logged in. A logout URL is amended to the claims map.
• This is an entirely stateless (modulo the current browser URL being a state) due to being repeatedly bit in the ass by various problems owing to Auth0 statefulness. Being stateless, it doesn’t store anything in cookies/localstorage, but contacts Auth0 for login on every page load, with the session being effectively as long as the Electric app is live. So no cookies/localstorage/refreshTokens etc. (yet). Being able to silently log in makes it bearable nevertheless.
• It makes use of some helpers from namespaces not included here. I’ve left a comment here and there, but if it’s unclear I can elucidate.#2024-01-1511:32danbuneaHad a brief look, you're using the Auth0 java API, I wanted to do the calls using http from the clojure server backend, so I can reuse them with other systems (directly Google or Azure AD, Keycloak, AWS Cognito). The basic redirection and exchanging the code for an access token then for the user info is quite similar. I just wanted all to have some unit tests.
Where we differ is the stateful vs stateless, and I think a stateless approach is better. I'd like to understand it better. Since I want to have the app running on iOS devices, this tends to cut off the websocket every time the app is sent into the background, so when it is active again it recreates the websocket, but I'd need to revalidate the login. Since the code isn't valid anymore and I have nothing on the browser how would I do it? Your Silent Login approach#2024-01-1511:33danbuneahow would it work?#2024-01-1512:05henrikThe flow is roughly:
1. Get a code
1.1 Get the code from query params of the current URL.
1.2 If that didn’t work, try a silent login. A silent login amounts to mounting an iframe and waiting for a webmessage to be passed back containing the code. This could fail inside the iframe, and it won’t be possible to know if or why, so a timeout is needed to cancel the attempt.
2. Use code or redirect
2.1 If there is a code at this stage, use it to exchange for claims etc.
2.2 If that didn’t work, redirect to the (non-silent, visible) login page. On return, the flow will re-run from 1.#2024-01-1512:15danbuneaI love your solution. My understanding is: In the iframe you're opening the login link, but since you already logged in before this will go directly to the redirect giving you a code which then you integrate in the normal flow#2024-01-1512:19henrikYeah, exactly. Auth0 itself keeps the user logged in, expiring the login according to the settings from their control plane etc. If the user has a session live over at Auth0, the silent login flow will return a code immediately.
This all happens very quickly, so there’s no reason to be stateful except if:
• You support offline usage (with Electric, we don’t anyway)
• The user has privacy plugins that block iframes. For such users, they’d encounter the visible login screen each time, which is why I’m considering (but not in a hurry) to have some statefulness.#2024-01-1512:23henrikEither way, starting out stateless, and then carefully adding state seemed like a better order of priorities.#2024-01-1512:25danbuneaIn fact the part I found the worse to do was the session part (using wrap-session in Ring). This avoids that part complely, as it relies of the stateful part of auth0's session cookies. As I said I love it. And I will change the code to use it.#2024-01-1400:28weii'm trying to scroll to a new item after it's been added. what's a good way to do this after the DOM updates?(see TODO)
#?(:clj (defonce !items (atom [])))
(e/def items (e/server (e/watch !items)))
#?(:clj
(defn add-item! [id]
(swap! !items conj {:item/id [id]})))
(e/client
(div
(ui/button
(e/fn []
(e/server (add-item! (util/uuid)))
;; TODO: wait for item to be added and fetch its dom node
(.scrollIntoView new-dom-node))
(text "Add an item")))
(e/for-by :item/id [{:item/keys [title]} items] (div title)))#2024-01-1400:51VincentHmm how could we watch for a dom addition :thinking_face:#2024-01-1409:51henrikWhen we do stuff like this, we usually stick the scroll logic in the item to be scrolled to itself. This is because race conditions become a non-issue, and because it’s the “correct” place in terms of colocation.
Of course, it requires some way of knowing that an item is new though, or it’ll run on every page load.#2024-01-1410:49Geoffrey GaillardIntersectionObserver and MutationObserver might help#2024-01-1418:54weiThat's a great idea! Will try it.#2024-01-1500:10weiworks! in the button handler, i add the item id to a set tracking new items:
(swap! !new-items conj new-id)
and the item handles the scrolling and removing itself from the new item set
(when (@!new-items id)
(when-let [node (.getElementById js/document id)]
(.scrollIntoView node))
(swap! !new-items disj id))
is there a way to reference the DOM node within its electric function, without resorting to getElementById?#2024-01-1507:48henrik(when (@!new-items id)
(.scrollIntoView dom/node)
(swap! !new-items disj id))
#2024-01-1507:50henrikIf you e/watch !new-items instead of dereferencing it, you effectively have a way to scroll to an item at any point in time, not just when initially adding them. Just a note for the future.#2024-01-1507:52henrikMake sure to e/watch once and pass down the value to the children in that case. Don’t e/watch once per child.#2024-01-1705:06wei@U06B8J0AJ would you mind elaborating on this e/watch !new-items being able to scroll to an item at any point in time? i thought about it a bit and still don't get it, but it seems interesting. maybe a (pseudo-)code snippet to illustrate?#2024-01-1707:33henrikSure! I only mean that scrolling doesn’t necessarily have anything to do with adding.
For example:
(e/defn Bar
[should-scroll]
(dom/div
(when should-scroll
(.scrollIntoView dom/node))))
(e/defn Foo
[]
(let [!scroll-to (atom nil)
scroll-to (e/watch !scroll-to)]
(e/for-by :id [{:keys [id]} [{:id 0} {:id 1} {:id 2}]]
(Bar. (= id scroll-to)))))
You can set !scroll-to to an ID and have it scroll for any reason. For example when adding an item, but you could also set it from the URL: my.app/?focus=1 or something, to support direct linking.#2024-01-1708:56weimakes sense, and that still circumvents the race condition issue from before. brilliant!#2024-01-1502:40VincentElectric is the juice#2024-01-1508:37Ken CCHi, I'm trying to have two variables (x, y) but
1. Only x will display.
2. Toggle button for y changes x???
#?(:clj (defonce !x (atom true))) ; server state
(e/def x (e/server (e/watch !x))) ; reactive signal derived from atom
#?(:clj (defonce !y (atom false)))
(e/def y (e/server (e/watch !y)))
(e/defn Hello2 []
(e/client
(dom/div (dom/text "X: " x))
(ui/button (e/fn []
(e/server (swap! !x not)))
(dom/text "toggle client/server x"))
(dom/div (dom/text "Y: " y))
(ui/button (e/fn []
(e/server (swap! !y not)))
(dom/text "toggle client/server y"))))
#2024-01-1508:51Ken CC#2024-01-1509:10Vincentvery close, (ui/button (e/fn []) (dom/props) (dom/text))
your ui/button is closed after e/fn [] and does not include the dom/text in the actual button
(ui/button
(e/fn []
(e/server (swap! !y not))
(dom/text "toggle y"))
#2024-01-1509:10Vincent2 parens after not, not 3#2024-01-1509:24Ken CCdom/text is inside of ui/button in my original post no?#2024-01-1509:38VincentOh it is lol))#2024-01-1509:39VincentHmmmm#2024-01-1509:41Vincent"the toggle button for y changing x" makes me think there is a paren missing or an invalid def somewhere else, but idk#2024-01-1509:41Ken CCit doesn't even show Y's value on screenshot. (It shows for X)#2024-01-1509:42Vincentdid you restart your server since adding the #?(:clj (defonce !y ..)) #2024-01-1509:44Vincentseems like it should work, based on the code you provided.#2024-01-1509:49Ken CCI just restarted, and it's still same.
I been stuck trying to get this to work for past 3 days, restarted multiple times, created variations to figure it out.#2024-01-1509:49Ken CCI'm running on WSL2. let me try running on Linux just incase#2024-01-1509:58henrikIf you flip the order: put the Y stuff before the X stuff, does Y still not display?#2024-01-1510:00henrikIf you change (dom/div (dom/text "Y: " y)) to (dom/div (dom/text "Y: " x)), does it display the value for x?#2024-01-1510:04Ken CCyes, that will show x.
Also it fixed button to toggle y. doesn't change x (correct behaviour)#2024-01-1510:18henrikWhat if you (dom/div (dom/text "Y: " (str y)))#2024-01-1510:18henrikI’m not sure how dom/text coerces to strings (if at all).#2024-01-1510:32Ken CCtried (str y), (pr-str y), no difference. y doesn't show up.
Also toggle button for y will change value for x again.#2024-01-1510:33Ken CCTried running on linux (manajaro) and no difference.
X shows, Y doesn't show.#2024-01-1511:11Daniel Galvincan make it work by wrapping
(dom/text "Y: " (e/server y))
but im not sure why it works yet 😕#2024-01-1511:13henrikOh… No, that makes sense actually.#2024-01-1511:15Daniel Galvinit confuses me because x seems to work without such wrapping. but to me it all appears identical#2024-01-1511:15henrikYou’re in a client block, hence you’re going to get the value of x/y in the client scope (which is nothing) by default, unless wrapped with e/server. What doesn’t make sense is why x is showing up without being wrapped.#2024-01-1511:17Ken CCwrapping in (`e/server y)` fixed it!!!#2024-01-1511:18Ken CCalthough unclear why x works as it is, and why Y button triggers X. (fixed once (e/server y) wrap is there)#2024-01-1512:03VincentIt might make more sense if you consider the defonce
(defonce !x #?(:cljs (atom false)) #?(:clj (atom false)))
in principle we ought define a clientside something and serverside something this way. if there is only one, you will have to specify which you are reading. might be a way to think about it/ reason about it#2024-01-1512:08henrikThis is all true, but it doesn’t quite explain why the server-side value was available from x. Could be a bug/glitch I suppose.#2024-01-1514:36Dustin GetzHi, please can you restate the problem crisply now that the workaround is understood? Possibly this is a new regression in the IC release#2024-01-1515:19Daniel GalvinIssue: Y value not rendering yet X value is
#?(:clj (defonce !x (atom true))) ; server state
(e/def x (e/server (e/watch !x))) ; reactive signal derived from atom
#?(:clj (defonce !y (atom false)))
(e/def y (e/server (e/watch !y)))
(e/defn Hello2 []
(e/client
(dom/div (dom/text "X: " x))
(ui/button (e/fn []
(e/server (swap! !x not)))
(dom/text "toggle client/server x"))
(dom/div (dom/text "Y: " y))
(ui/button (e/fn []
(e/server (swap! !y not)))
(dom/text "toggle client/server y"))))
In the code snippet you see there are 2 atoms defined
#?(:clj (defonce !x (atom false)))
#?(:clj (defonce !y (atom false)))
these have watch set up on them via e/server like so
(e/def x (e/server (e/watch !x)))
(e/def y (e/server (e/watch !y)))
in the Hello2 fn, we are defining some divs with some text and a couple ui buttons
this is all wrapped in (e/client)
when ran, the div are rendered. but there is no value given to the second div for 'y'
however the first div does succesfully render 'x'
By wrapping 'y' in (e/server ..) we can get it to work correctly.
The confusion is why 'x' was working without the need to be wrapped in (e/server ..)#2024-01-1515:20Dustin GetzCan you make it complete/self-contained please so I can put this in an issue tracker and pass it on#2024-01-1515:23Daniel Galvinhave updated above to contain the full snippet aswell as a breakdown of steps as to why its weird, along with the fix done#2024-01-1515:23Dustin Getzthanks, is it minimized?#2024-01-1515:25Daniel Galvinnot sure, its pretty concise apologies if not that useful i was not original raiser. just had a look at resolving it to help.#2024-01-1515:27Dustin Getz~ok, i see, we agree that ~ [this is incoherent sorry] This looks like a regression to me, I also think x access needs to be wrapped in Electric v2#2024-01-1515:27Dustin Getzthanks#2024-01-1515:30Dustin Getz~I expect accessing ~
this is incoherent sorry#2024-01-1515:41VincentI am wondering if defining the y atom to true would make y also glitchy like that#2024-01-1515:47Ken CCI basically copied "2. Toggle" example in fiddle, and tried to modify slightly (add another variable). if it is case x should be wrapped, fiddle example also should be changed?#2024-01-1515:48Dustin Getzplease link to the thing you changed?#2024-01-1515:49Ken CCyep, sec doing it#2024-01-1515:52Ken CC2 problems.
a. y will not show.
b. pressing button to change y's state will change x.
#?(:clj (defonce !x (atom true))) ; server state
(e/def x (e/server (e/watch !x))) ; reactive signal derived from atom
#?(:clj (defonce !y (atom false)))
(e/def y (e/server (e/watch !y)))
(e/defn Hello2 []
(e/client
(dom/div (dom/text "X: " x))
(ui/button (e/fn []
(e/server (swap! !x not)))
(dom/text "toggle client/server x"))
(dom/div (dom/text "Y: " y))
(ui/button (e/fn []
(e/server (swap! !y not)))
(dom/text "toggle client/server y"))))#2024-01-1515:52Ken CC#?(:clj (defonce !x (atom true))) ; server state
(e/def x (e/server (e/watch !x))) ; reactive signal derived from atom
#?(:clj (defonce !y (atom false)))
(e/def y (e/server (e/watch !y)))
(e/defn Hello2 []
(e/client
(dom/div (dom/text "X: " x))
(ui/button (e/fn []
(e/server (swap! !x not)))
(dom/text "toggle client/server x"))
(dom/div (dom/text "Y: " (e/server y))) ; props had a [node m] arity? The scenario is that a child needs to flip a switch on the properties of an ancestor. The line of thought is: what if instead of passing these props from child to the ancestor, I pass the dom/node to the child?#2024-01-1720:21Geoffrey GaillardThere are no technical limitations. But is it the right thing to do design-wise? How are you achieving it today?#2024-01-1720:36henrikIn this particular scenario, I copied props and modified it to try out the pattern to see how it feels. In other similar situations, I guess I would use a return value (which would get complex in this case since it would have to pass multiple intermediate boundaries, making it kind of fragile), or go via a mutable state: bind an atom that the child can use to signal information to the ancestor.
I’m also not sure if it’s right, but the following thoughts occurred that might contribute something:
1. Props are allowed to cross function boundaries to the direct parent DOM node, so why not generalize “parent” to “ancestor”?
2. on and on! have node arity, and they semantically modify props. Why should they be special?#2024-01-1914:16Geoffrey GaillardAgreed 👍
We’ll give it a look#2024-01-2315:41Geoffrey GaillardAvailable on master: https://github.com/hyperfiddle/electric/commit/91cc34aa9c43e0454b0f7f89365519959313b8a5#2024-01-2315:46henrikNice, thank you!#2024-01-1718:18hadilsHas anyone experimented with electric on react native?#2024-01-1801:34VincentI have need for a custom playback button for an audio element.#2024-01-1801:34VincentAnd just wanted to let you all know)#2024-01-1802:19Vincent#?(:cljs (defn init-audio []
(.log js/console "[init-audio...]")
(let [audio-element (new js/Audio "uploads/fits-together.m4a")]
(.load audio-element)
(.play audio-element))))
works...
exciting...#2024-01-1812:31pezHas anyone used RCF tests with #C015LCR9MHD? JS SCI is where I want it, and just wanted to check if I should even bother try.#2024-01-1812:32borkdudeIn what sense, explain more about the context#2024-01-1812:34pezI’m working with a rather complex Joyride app, and I like the way RCF lets me express my tests similar to rich comment forms.#2024-01-1812:36pezI’ll try drop rcf in to the project and see what happens. But if someone else has already tried, I would love to know.#2024-01-1812:37borkdudercf does work with bb so it might work with joyride too, not sure#2024-01-1812:47pez> ; No matching clause: require-macros
When trying to require hyperfiddle.rcf. I can’t dig into it right now, but will try to investigate later.#2024-01-1812:48borkdudetry a newer version of SCI#2024-01-1812:49borkdudehttps://github.com/babashka/sci/commit/348f6839495fe71f8f3b49680f12e95577d822a5#2024-01-1812:49borkdude(but do use the newest version)#2024-01-1812:53pezAh, yes, we do use a rather old version in Joyride: https://github.com/babashka/sci/blob/39ce36540eb4c2c6adc74c23ea76ac6330ca7835/CHANGELOG.md#2024-01-1823:43pezWith latest SCI I get:
; Cyclic load dependency: [ hyperfiddle.rcf ]->[ hyperfiddle.rcf ]
Evaluation of file util.cljs failed: #error
{:cause
#error
{:data
{:column nil :file "hyperfiddle/rcf.cljc" :line nil :type :sci/error}
:message
"Cyclic load dependency: [ hyperfiddle.rcf ]->[ hyperfiddle.rcf ]"}
:data
{:column 3
:file "hyperfiddle/rcf.cljc"
:line 3
:message "Cyclic load dependency: [ hyperfiddle.rcf ]->[ hyperfiddle.rcf ]"
:sci.impl/callstack
#object
[cljs.core.Volatile
{:val
({:column 3
:file
"/Users/pez/Projects/openai/kurshowcrisp/.joyride/src/backseat_driver/util.cljs"
:line 2
:ns #object [sci.lang.Namespace]}
{:column 3
:file "hyperfiddle/rcf.cljc"
:line 3
:ns #object [sci.lang.Namespace]})}]
:type :sci/error}
:message "Cyclic load dependency: [ hyperfiddle.rcf ]->[ hyperfiddle.rcf ]"}#2024-01-1823:44pezThe ns form of hyperfiddle/rcf.cljc:
(ns hyperfiddle.rcf
(:refer-clojure :exclude [=])
#?(:cljs (:require-macros [hyperfiddle.rcf :refer [tests deftest async]]
[hyperfiddle.rcf.impl :refer [make-queue]]))
(:require #?(:clj [hyperfiddle.rcf.impl :as impl])
#?(:clj [clojure.test :as t]
:cljs [cljs.test :as t])
#?(:clj [hyperfiddle.rcf.analyzer :as ana])
#?(:clj [clojure.walk :as walk])
#?(:clj [ :as io])
[clojure.string :as str]
[hyperfiddle.rcf.reporters]
[hyperfiddle.rcf.queue]
[hyperfiddle.rcf.time]
[hyperfiddle.rcf.unify :as u]))#2024-01-1909:01borkdudehm yeah I guess rcf isn't written to be used in a self-hosted-like cljs environment. Is there anything we can do about that in the :load-fn ourselves in joyride?#2024-01-1813:39JHi guys! This error talks to someone?#2024-01-1815:43JMy fault! Missing a cljs block#2024-01-1821:31john-shafferTracked down some regressions in the latest v2-alpha-540-ga4699532 release with jetty:
• The old adapter was https://github.com/hyperfiddle/electric/blob/4b543d46ab410c8f1cee9c006cd35eecbd973c99/src/hyperfiddle/electric_jetty_adapter.clj#L59. The new adapter uses the default of 65536, which is much too low for electric.
• The default for the websocket idle timeout seems to be 30 seconds, but the heartbeat is sent at https://github.com/hyperfiddle/electric/blob/d41e0c18d5914441e276df69843a7bd9116cf8b7/src/hyperfiddle/electric_ring_adapter.clj#L19C8-L21. This quickly results in a server-side error and the browser page going blank.
I am working around by monkey-patching ring.adapter.jetty/upgrade-to-websocket
(defn- upgrade-to-websocket [^HttpServletRequest request response response-map]
(let [context (.getServletContext request)
container (JettyWebSocketServerContainer/getContainer context)
creator (websocket-creator response-map)]
(.setIdleTimeout container (java.time.Duration/ofSeconds 120))
(.setMaxTextMessageSize container (* 100 1024 1024))
(.upgrade container creator request response)))
#2024-01-1912:53Dustin GetzWe think you're using an old jetty boilerplate, see breaking changes listed in the Electric https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#2024-01-1912:54Dustin Getzit links to jetty and httpkit examples which are properly configured#2024-01-1915:21john-shafferThanks! I just needed the configure-websocket! bit from the electric-fiddle repo and it's working great again#2024-01-1822:13simonkatzI’m trying to call some electric code from the CLJS REPL.
I have this ns in my_ns.cljc:
(ns my-ns
(:require [hyperfiddle.electric :as e]))
(e/defn foo []
(e/server (println 42)))
I think I’ve seen that I should be able to call this from a CLJS REPL using:
(hyperfiddle.electric/run (my-ns/foo))
but when I try that I get the following:
------ WARNING - :undeclared-var -----------------------------------------------
Resource:
Is there a way to call my-ns/foo, either from a CLJS or CLJ REPL?#2024-01-1822:33simonkatzHmmm. I’ve just realised that (I think) all the examples of e/defn I’ve seen define UI things (so (e/defn Foo [...] ...) , called with (Foo. ...) .
So maybe that’s at least part of the issue.
What I’m really trying to is run some server code from the CLJS REPL, to verify that the CLJ REPL I have in my dev env (Emacs) is the same one that my client code is talking to. (I think it might be different.)#2024-01-1822:39simonkatzI’ve verified that I do indeed seem to have two values for the same var in clj — I’m now displaying its value in the UI and seeing a different value to what I see when I evaluate it in Emacs.
Things match up until I do a cider-ns-refresh in Emacs, but then things go weird.#2024-01-1822:45simonkatzAh, maybe because namespace-refresh is creating a brand new namespace, and electric is using the old namespace of the same name.#2024-01-1822:47simonkatzI’ve recently switched to playing with electric-fiddle (https://github.com/hyperfiddle/electric-fiddle).
Previously I was using electric-starter-app (https://github.com/hyperfiddle/electric-starter-app), and that was OK with namespace-refresh.#2024-01-1822:53simonkatzI said electric-starter-app was OK with namespace-refresh, but that’s with a couple of provisos — (1) I had to delete the app.electric-server-java11-jetty10 namespace because it wouldn’t compile, and (2) it didn’t work with the latest commits (on the legacy branch — I had to revert to an earlier commit).#2024-01-1822:55simonkatzSorry for the dribs and drabs.
My question is: can namespace-refresh work with electric-fiddle ?
EDIT: I should explain, I have some .clj files in my project for server-side code.#2024-01-1823:53Dustin Getzi’m not the right person to answer this but in the meantime you can delete all the extra fiddle loader stuff from electric fiddle and just use it in the same minimalist configuration as the starter app. The only reason we retired starter-app is because the boilerplates kept getting out of sync#2024-01-1823:55Dustin Getzactually i think cider-refresh-ns is not enough in Electric because you need the client to rebuild transactionally with the server#2024-01-1823:56Dustin Getzif it worked before, perhaps this is due to having an older Electric version prior to the new Incremental Compilation release? IC changeset changed how the client and server build#2024-01-1912:16simonkatz> if it worked before, perhaps this is due to having an older Electric version prior to the new Incremental Compilation release
Yes, that’s it. I’ve checked in my electric-starter-app-based project, and the introduction of incremental compilation causes the bad interaction with clojure.tools.namespace.
Might it be possible to make that work again? Perhaps as an option, if, for example, it makes things slower.#2024-01-1912:25Dustin GetzWe didn't know, I will log a ticket#2024-01-1912:28simonkatzGreat; thank you#2024-02-0611:13simonkatzJust in case this has slipped between the cracks…
I didn’t notice a ticket for this issue with clojure.tools.namespace (looking at https://github.com/hyperfiddle/electric/issues?q=is%3Aopen+is%3Aissue).
Was there a suggestion that electric v3 would fix this issue? (in the thread at
https://clojurians.slack.com/archives/C7Q9GSHFV/p1706692314775209)#2024-02-0611:18xificurCit's logged in our internal ticket db#2024-02-0611:19simonkatzOK; thank you#2024-01-1822:55simonkatzSorry for the dribs and drabs.
My question is: can namespace-refresh work with electric-fiddle ?
EDIT: I should explain, I have some .clj files in my project for server-side code.#2024-01-1823:36VincentI want to do a "toaster" for new notifications when people add a new song to the Shomp inventory#2024-01-1823:36Vincentwrite to db will write to a notifications atom with a timestamp...#2024-01-1823:37Vincentis there a way i can have the notifications come in staggered tempo // time-wise come in slightly apart from one another if they are "historical" ? real-time notifs are cool, but ~previous 30 min of notifs will [barrage user all at once] without some sort of staggered replay functionality#2024-01-1823:38Vincentwondering what the brilliant fellows of this room might think about it#2024-01-1823:47Vincentfor starters, i asked chappy4 to work off of my notes and it created a sorta good result in my eyes:
the main hangup/question i had was how to js/setTimeout conj notifs but it seems like it can work now. Will this work in sequence as desired?
(e/defn toaster []
(e/server
(let [now (System/currentTimeMillis)
active-notifs (filter ... now ts x notifs)] ;; Assuming this filters the active notifications
(e/client
(let [display-notifs (r/atom [])]
(doseq [notif active-notifs]
(js/setTimeout
(fn [] (swap! display-notifs conj notif)) ;; Adds each notification to the display list
(* 1000 delay-in-seconds))) ;; delay-in-seconds controls the staggered display
(for [notif @display-notifs]
;; Render each notification
[:div ... ]))))))#2024-01-1823:50Vincentside musings >#2024-01-1823:50Vincentas an aside:
authoritative ways to do clientside cookie save + validate on every "action" [1]
and some sort of baked-in toaster solution [2]
would be very nice for every single one of my electric projects (three so far)
not exacttttly sure what those look like but off the top of my head i think prior attempts to solve/resolve the login procedure is to label un-logged-in people as guests ,, then have users and admins and do varying case role :user (user casserole? ) statements before every action? seems sorta... unwieldy. i basically want a 'you're logged in, go ahead and mutate the database' and 'you're not logged in, but everyone should get a visual cue you're here and no you should not modify the db'#2024-02-0611:13simonkatzJust in case this has slipped between the cracks…
I didn’t notice a ticket for this issue with clojure.tools.namespace (looking at https://github.com/hyperfiddle/electric/issues?q=is%3Aopen+is%3Aissue).
Was there a suggestion that electric v3 would fix this issue? (in the thread at
https://clojurians.slack.com/archives/C7Q9GSHFV/p1706692314775209)#2024-01-1916:56VincentMade a big code push to prod and ...#2024-01-1916:58Vincentthe page renders on the desktop, but on iphone (connected to mac via developer console...)#2024-01-1916:59Vincentgotta roll back bit by bit i guess x_x#2024-01-1917:08Geoffrey GaillardAre you running latest electric?
The RangeError: Maximum call stack size exceeded. error has been fixed a while ago#2024-01-1918:00VincentOh good ). maybe just out of date#2024-01-1918:20VincentMust I do something new to expose e/*http-request ?*#2024-01-1918:26VincentLooks like e/http-request became
e/http-request
which is fine, but took me some sorting out x). the compile assist notes are nice although unaware of previous version naming conventions#2024-01-1918:29xificurCSee the https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#v2-alpha-536-g0c582f78--2024-jan-10 for what changed#2024-01-1918:30Vincentoh thank you forgot those existed#2024-01-1918:29VincentI like the :toggle a lot -- that says if it's client or server? one of my things thinks it's client but it's def a serverside fn...#2024-01-1918:30VincentError in phase :execution
in Art-Description-Brisket: (generate-album-art (:hyperfiddle.electric.impl.lang/toggle :client {:line 329, :column 60, :hyperfiddle.electric.debug/type :transfer, :hyperfiddle.electric.debug/name :hyperfiddle.electric/client} (clojure.core/deref !!art-desc)))
I cannot resolve `generate-album-art`, maybe it's defined only on the client?
but generate-album-art is
#?(:clj
(defn generate-album-art
[prompt]...))
#2024-01-1918:30xificurCIs it defined later? Order now matters#2024-01-1918:33Vincentlet me see. have been ignoring line numbers since may#2024-01-1918:33Vincent🍪 you get a cookie#2024-01-1918:37Vincentokay looks like a lot changed and i might be better off adding components one by one to the latest and greatest....#2024-01-1918:37Vincentnot sure if the change around binding e/http-request needing ring-req makes it different enough#2024-01-1918:47VincentOkay I think I need to add (e/client immediately inside all my e/defns...
and make sure all the clj serverside asks are above their invokes...#2024-01-1918:51Vincentand it looks like all functions in electric namespaces have to be exposed to both client/server even if one is not used... which is okay#2024-01-1918:55Vincentlol yeah looking like i'm gonna have to start from close to fresh for this one xD#2024-01-1919:03Vincenthappiness level with migrating between old and breaking changes... local minima discovered 😉#2024-01-1919:04VincentUpgrading to the latest ...#2024-01-1919:04VincentExecution error (FileNotFoundException) at app.electric-server-java8-jetty9/eval62161$loading (electric_server_java8_jetty9.clj:3).
Could not locate hyperfiddle/electric_jetty_adapter__init.class, hyperfiddle/electric_jetty_adapter.clj or hyperfiddle/electric_jetty_adapter.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
happens presumably because my app is not passing in ring-req alongside?#2024-01-1919:06VincentNot sure where to / how to dougie
(e/defn FiddleMain [ring-req]
(e/client
(binding [hf/pages fiddles]
(e/server
(electric-fiddle.main/Main. ring-req)))))#2024-01-1919:33xificurCIf you upgraded to latest check the ring changes too. Above the IC change log#2024-01-1919:39Vincentholding the cpu at gun point is also not helping#2024-01-1919:46Vincentno prob,.. gonna pull the new demos down and work from that i suppose. src-fiddles ?#2024-01-1919:51VincentRing and middleware are a personal weak spot of mine 苦手です。 so the mission to fix a small deviation i didn't invent is sapping a lot of energy haha but it's okay i'm on the mission to freshen up the repos so... here we go)#2024-01-1920:04xificurCsorry for the inconvenience, unfortunately with bleeding edge tech we sometimes have to commit breaking changes#2024-01-2000:59Vincentnot a problem, i can sulk in private )#2024-01-2000:59Vincenti do have needs for: an app that supports xtdb and chat/presence ootb that i can extend into prod versions of apps#2024-01-2019:53Vincentanyone else having trouble cloning git clone and that worked. but going via github desktop link on git did not ;x#2024-01-2020:00Dustin Getztry the ssh url#2024-01-2020:00Dustin Getzgit@ not http#2024-01-2020:05Vincentwhy does git even have an http url there -_-#2024-01-2020:07VincentThanks)))#2024-01-2021:08Vincentwhat is yarn for?#2024-01-2021:10Dustin Getzyarn is same as “npm install” and i think optional#2024-01-2101:43VincentAh ok#2024-01-2021:05braai engineerDid hyperfiddle.electric/boot go away in latest version?#2024-01-2021:24Geoffrey GaillardYes, it's been replaced by e/boot-client and e/boot-server. See the https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#2024-01-2100:20anonimitorafHi guys, I’ve been trying to make modular components and have been encountering some problems with callbacks. I get this error:
"clicked"
#object[TypeError TypeError: n.call is not a function]
Code:
(e/defn BasicOption
[{:keys [on-select]}]
(e/client
(dom/li (dom/text "Pick me")
(dom/on "click"
(e/fn [_]
(prn "clicked")
(on-select "me"))))))
(e/defn BasicSelect
[{:keys [on-select] :as props}]
(e/client
(dom/ul
(BasicOption. {:on-select (e/fn [selected] (on-select selected))}))))
(e/defn MySelect
[]
(e/client
(BasicSelect. {:on-select (e/fn [selected] (prn "selected" selected))})))#2024-01-2100:21anonimitorafI’ve also encountered scenarios where I get:
#object[Error Error: Invalid arity: 0]
Happy to try make a minimal repro for it later#2024-01-2100:30Dustin Getze/fn must be called with new, i think that is “n.call is not a fn”#2024-01-2100:31Dustin Getzon-select is e/fn but called like cljs fn. We recommend capitalizing your electric lambdas to help remember (like react components)#2024-01-2100:33anonimitoraf called with newLike this?
(e/defn BasicOption
[{:keys [OnSelect]}]
(e/client
(dom/li (dom/text "Pick me")
(dom/on "click"
(e/fn [_]
(prn "clicked")
(OnSelect "me"))))))
(e/defn BasicSelect
[{:keys [OnSelect] :as props}]
(e/client
(dom/ul
(BasicOption. {:OnSelect (new (e/fn [selected] (OnSelect selected)))}))))
(e/defn MySelect
[]
(e/client
(BasicSelect. {:OnSelect (new (e/fn [selected] (prn "selected" selected)))})))
With that^, I get this error:
#error {:message You called #2024-01-2101:19Dustin Getz(OnSelect. x)#2024-01-2103:40anonimitorafI see, thanks!
(e/defn BasicOption
[{:keys [OnSelect]}]
(e/client
(dom/li (dom/text "Pick me")
(dom/on "click"
(e/fn [event]
(prn "clicked!!!!" event)
(OnSelect. "me"))))))
(e/defn BasicSelect
[{:keys [OnSelect] :as props}]
(e/client
(dom/ul
(BasicOption. {:OnSelect (e/fn [selected] (OnSelect. selected))}))))
(e/defn MySelect
[]
(e/client
(BasicSelect. {:OnSelect (e/fn [selected] (prn "selected" selected))})))#2024-01-2102:28VincentAnother question, when minting a prod asset, which index.html is it gonna use?
I have yet to experiment with minifying the app since it's yet to be functional enough to deem minification,
but i'm wondering ... it needs a streamlined index.html yes?#2024-01-2104:44VincentI have no idea how to add xtdb to the latest and greatest instantiation of hyperfiddle electric. In the xtdb starter there's a user.clj namespace that presumably auto-loads (just something Clojure does?) and ... so...
what do I do#2024-01-2113:38Dustin Getzyes user.clj is auto loaded by Clojure#2024-01-2113:38Dustin Getzyou can copy this xtdb code into your project https://github.com/hyperfiddle/electric-xtdb-starter/blob/master/src/user.clj#2024-01-2113:40Dustin Getzin electric-fiddle we use https://github.com/hyperfiddle/electric-fiddle/blob/main/src-dev/dev.cljc and https://github.com/hyperfiddle/electric-fiddle/blob/main/src-prod/prod.cljc due to limitations and issues with how clojure loads user.clj (for example, no .cljc support, we wanted cljs and clj entrypoints to be at the same place, we also are very careful to keep dev/prod classpaths isolated to prevent later issues in uberjar builds)#2024-01-2113:40Dustin GetzI don't understand your specific problem, this is all just Clojure#2024-01-2113:42Dustin Getzyou don't have to use electric-fiddle either, it is just an example of how to set up an entrypoint for you to use. You simply must understand how the entrypoint works, there is no way around it#2024-01-2116:15VincentI think the entrypoint is mainly to bind e/http-requst to ring-req ... i'm missing something fundamental about getting XTDB back into the equatio#2024-01-2116:26Vincentthanks maybe dropping in the user.clj was the missing piece#2024-01-2116:26Vincent"find the difference in these 2 images" at midnight always fun lol.#2024-01-2116:29VincentI drop user.clj into the folder and get "I cannot resolve user/!xtdb, maybe it's only defined on the client? Something special I have to do to get the new deployment scheme to load the user namespace?#2024-01-2116:34Vincentreally just wanting to move my project to latest electric version, maybe i just need to honor the new entrypoint, but i thought it would be good to update the scaffolding if you fine chaps are gonna keep improving it this way going forward, since any scaffolding change will be a little bandaid pull sooner or later#2024-01-2117:07VincentBasically wanting to: update entry point and use XTDB ... migrating is kicking my butt#2024-01-2119:03Dustin GetzTo close the loop here I got Vincent sorted over zoom, and we should have an xtdb starter app soon#2024-01-2119:06VincentDustin is a gentleman and a scholar and really helped me get through my issue today, thanks man.#2024-01-2119:46Vincenthmm, when I re-save a file it wants to rebind !xtdb to (start-xtdb!) and ends up saying no Lock available. if I invoke (.close !xtdb) later it will work okay, but then it's basically turning the db off after a query? ..pondering#2024-01-2119:48Dustin Getzyou can use m/observe to manage the lifecycle of that connection (auto close it on app shutdown) - search m/observe for past discussions #2024-01-2119:51Vincentokay cool! i gotta learn thot.#2024-01-2119:51Vincentby the way, the version i was using before moving to the latest and greatest was com.hyperfiddle/electric {:mvn/version "v2-alpha-422-g96e5c0a5"} and it appears to have IC thot doesn't care too much about sequentiality?#2024-01-2120:02Vincentnvm, not IC based on changelog.#2024-01-2120:16VincentOh yeah that m/observe exists in the db/latest-db> method from xtdb-contrib, but i'm having trouble binding it#2024-01-2120:33Vincent(e/defn FiddleMain [ring-request]
(e/server
(binding [e/http-request ring-request ; make ring request available through the app
!xtdb (start-xtdb!)
db (latest-db> !xtdb)]
(e/client
(binding [dom/node js/document.body] ; where to mount dom elements
(Hello.))))))
#2024-01-2120:33Vincent... new xtdb starter coming later? heard.
vacation mode#2024-01-2119:04Dustin GetzQuick straw poll: For those who tried to clone and run electric-fiddle: did you get it to work?#2024-01-2120:08teodorluI haven’t used electric in a few months, but I was able to get started quickly without any issues. Followed the README, things worked as I expected.
Connecting a REPL went smoothly too. Perhaps that could be more explicit. Depends on your audience, I guess, you’re not doing anything weird for someone used to shadow-cljs.#2024-01-2206:59braai engineerHi Dustin, plz consider having a simple electric-starter that just does one app. Users like me want one app that does one thing that can be easily deployed without stripping stuff out (esp. if you're not sure what you can strip out safely).#2024-01-2219:29NikolaiMaybe this is off-topic, but is there anyone who uses VS Code with Calva, find out that the ClojureDocs integration does not work properly for Missionary in fiddles? (Or it is just my wrong setup). In the electric-starter-app it was working as expected, e.g. mouse pointing to 'm/..' showing popup with the doc strings.#2024-01-2517:28Ken CCyes for me. my clone is from this year but not the latest version#2024-01-2611:28Geoffrey GaillardWe’ve updated Electric Starter App (@U051SPP9Z)
https://github.com/hyperfiddle/electric-starter-app#2024-01-2220:18henrikThis is a spike/prototype/pre-alpha rich text editor, written from scratch using Electric, Automerge, and Rama.
I built it to prove (to myself) that:
• Automerge’s capabilities are complete enough that I should be able to express a rich text editor entirely in terms of transformations on an Automerge document. I.e., there’s no intermediary state other than that which the AM document holds, and the editor can be expressed with view=f(data), where the automerge document is the data part. This turned out partly true, because in the end I didn’t want to serialize the cursor positions into the document, so they are side-channeled.
• Electric is capable of being the f part in the same equation, producing the correct event ordering, semantics, etc. necessary in order to keep the view updated a) fast enough, and b) consistent enough that there are no glitches with cursors, selections, etc., even when absolutely hammering the keyboard like a squirrel on acid (and also when being a collaborator on the receiving end of the output of aforementioned squirrel).
• Rama can act as a document repository (something that otherwise requires specialized infrastructure, and has certain problems, such as with scaling); receiving a stream of changes and (exactly once, in the correct order) batch those changes and atomically update a document stored on disk, replicate to n nodes, and keep m clients up-to-date.
Additionally,
• There should be a way to render “local-only” objects in the editor (menus, input-boxes, whatever).
• There should be a way to express simple and complex object-like things (images, tables, etc.), in the editor and document state.
I’ve made no consideration of performance other than in the “big O” sense. I.e., rendering latency should not exceed linearity to the size of the content visible on screen.
The footprint of the codebase turned out like this:
• Code for turning an AM document into renderable data structures, ~200 lines of code.
• Code for rendering DOM elements and translating DOM actions into appropriate actions on the Automerge document, ~300 lines of code total.
• Rama module for updating documents, ~60 lines of code plus another ~50 various helper functions.
• Code for syncing cursors and document content (client/server), ~100 lines or so.
This is not including another ~250 lines of code for just exposing a Clojure CLJC for Automerge that was symmetrical for both backend and frontend. I’m probably not accounting for helpers that have been in the codebase forever, and that I take for granted.
I can’t say much about how long time this took, since I wrote 90% of it during Christmas vacations as a fun way of torturing myself, and then stalled on some misunderstandings and bits I was missing in the Automerge API (which they very promptly took on board and amended, or otherwise helped me along).
Note: this is feature-incomplete, lacks tests, rigour, consistency and conciseness, but at least satisfies the goals I set out for it.
I’m looking forward to seeing how my thinking around this changes with the forthcoming differential Electric (it’s already changed a bit).#2024-01-2221:24VincentAmazing and very inspiring -- love that you pulled in an AI agent#2024-01-2300:05weithis is slick! any plans to publish the code?#2024-01-2300:06weithanks for consistently pushing the envelope forward - i've learned a lot from reading your code#2024-01-2308:01henrikThanks!
I wouldn’t say “plans” as much as “thoughts”. It’s nowhere near a releasable state by any measure, and certainly not to the standards I would expect from a Clojure library.#2024-01-2309:08weia library would be amazing, but it could be a valuable reference project even in its current state#2024-01-2309:51henrikSorry, but no. In its current state, it’s not representative of something I want to put into the world.#2024-01-2221:24VincentAmazing and very inspiring -- love that you pulled in an AI agent#2024-01-2309:20KetanAre there instructions somewhere on how to add electric to a project? I'm trying to follow electric-fiddle but just wanted to check if there's an easier way#2024-01-2415:48Geoffrey GaillardWe are working on a minimal ("brutalist") starter app that contains the bare minimum to run an electric program. We want it to be a reference on the minimum things one need to copy over an existing project.#2024-01-2611:25Geoffrey GaillardWe have a new starter app. The readme should help you integrate into an existing project.
https://github.com/hyperfiddle/electric-starter-app#2024-01-2320:09weiis there a good drag and drop library that works well with electric? something like https://shopify.github.io/draggable/ that can do sorting, swapping, drop targets, etc. the problem with that one is it wants to manage the DOM too, so i'm assuming it's incompatible#2024-01-2321:05grounded_sageI don’t see anything in here that wouldn’t be take much effort to do from scratch. My experience is most JavaScript libraries just save people from having to write a bunch of JavaScript and it takes very little code when done in Electric#2024-01-2407:53henrikI used https://dndkit.com/ in the UIX version of our app and got pretty familiar with it. While I agree that basic DnD isn’t hard in Electric, some of the more advanced stuff isn’t so straightforward. Like https://master--5fc05e08a4a65d0021ae0bf2.chromatic.com/?path=/story/examples-tree-sortable--all-features
I think the optimistic stuff in DE will be a great foundation for building a general DnD solution though.#2024-01-2418:42weiwas your app react+electric?#2024-01-2418:45weiwhat are the considerations for using react in an electric app? is there a large performance hit? i've avoided it so far since i didn't want to introduce another dependency. but the react ecosystem has a lot of useful, nontrivial libraries.#2024-01-2507:31henrikNo, the old app was straight up UIx.
I wouldn’t code large parts of the app in React, because Electric is better. But for lifting a React component off the shelf and using it, it’s fine, both usage-wise and performance-wise.#2024-01-2507:34henrikFWIW, this is the macro we use to mount React components with UIx. Usage is like:
(uix CircularProgressbar
{:value …
:className …
:styles …})#2024-01-2700:58weithanks for your notes on usability and the macro, i think it will be very useful. i'm trying to get this test running (your code is under app.util.uix):
(ns app.test
#?(:cljs (:require-macros [app.util.uix :refer [uix]]))
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :refer [div]]
#?(:cljs ["@heroicons/react/20/solid" :as heroicons])))
(e/defn Test2 []
(e/client (div (uix heroicons/ChevronUpDownIcon {}))))
which compiles but produces this js error:
#object[TypeError TypeError: Cannot read properties of undefined (reading 'create_root')]
what am i missing? i don't know which create_root it's referring to and there's no stacktrace. apologies if this is an elementary question but i'm new to UIx.#2024-01-2708:59henrikAh, weird. Are you on the latest maven artifact/master? This macro took a considerable amount of tweaking to get it to compile under IC.#2024-01-2709:34henrikOh, also, try [app.util.uix :as uix] and uix/uix.#2024-01-2715:29weithat worked, thanks! turns out i didn't need require-macros, and i did need other non-macro functions.#2024-01-3016:52weiwhen using something like UIx, what's a good way to call an e/fn from a React component? for example, I'd like to use electric to update some server atom when the user interacts with the component.#2024-01-3017:15henrikYou can only call Electric indirectly using a state machine from within a React component. The flow would be something like this:
1. Create an atom outside of the React component. e/watch it.
2. Create a switching mechanism that reacts to the state value, e.g. with when/`cond`/`condp`.
3. Create a Clojure (non-Electric) function that sets the state. Provide it to the React component.
Now your React function can set the state of the atom, and Electric will react to it.#2024-01-3017:22weithanks, makes sense. i also found a janky way to do react-component->electric by dispatching custom events within the component and using dom/on to listen for them#2024-01-3017:25weimy problem with #3 is you need electric to set a server-side atom (unless you want to handle network manually like the bad old days)#2024-01-3017:29henrikNot necessarily, consider this:
(e/defn Something
[]
(e/client
(let [!state (atom nil)
[action payload] (e/watch !state)]
(condp = action
:save (e/server (transact! db payload …) …))
(uix/uix ui-some-component
{:save-fn (fn [evt] (reset! !state [:save (.. evt -target -value)]))
…}))))#2024-01-3017:30henrikWrote this off the cuff, so there may be mistakes, but the principle is there.#2024-01-3017:30weii see, that's cool!#2024-01-2401:15joshchohas anyone experimented with electric and crdt’s?#2024-01-2408:07henrikThe RT editor above uses a CRDT (Automerge). AMA.#2024-01-2408:08joshchooh neat!#2024-01-2408:08joshchoi noticed in the demo that the changes weren’t character-by-character, is that dependent on how you set it up?#2024-01-2408:08joshchomerging, etc.#2024-01-2408:12henrikThe reason it isn’t character by character is because I roundtrip to Rama between. Automerge can be used peer-to-peer as well, or just against an in-memory document that you keep server-side. However, I 1) didn’t want to send out changes that to peers unless I know they have been durably recorded, and 2) If you go for the in-memory document approach, you get scalability problems. Rama takes care of scalability.
So for my approach, I went for a microbatch topology, meaning that Rama will collect 200-400ms worth of changes coming in, aggregate and save them, and only then will those changes be pushed to peers.
You could set it up for quick turnaround times, but I went for scalability and consistency instead.#2024-01-2408:14joshchowoah thanks for the detailed response! this is awesome :saluting_face: #2024-01-2407:51weihas anyone seen shadow OOM errors like this? seems to happen to me every few compilations. I've tried giving the shadow more memory (`-Xmx5G`) to no avail. I do have a 500+ line cljc file, but I've seen shadow work with larger files. I'm on the IC branch. wondering how to further troubleshoot this.
[2024-01-23 23:41:50.852 - WARNING] :shadow.cljs.devtools.server.util/handle-ex - {:msg {:namespaces #{app.scopes4}, :deleted #{}, :updated #{app.scopes4}, :added #{}, :macros #{}, :shadow.cljs.model/topic :shadow.cljs.model/resource-update}}
OutOfMemoryError Java heap space
java.util.Arrays.copyOf (Arrays.java:3537)
java.lang.AbstractStringBuilder.ensureCapacityInternal (AbstractStringBuilder.java:228)
java.lang.AbstractStringBuilder.append (AbstractStringBuilder.java:582)
java.lang.StringBuilder.append (StringBuilder.java:179)
clojure.string/join (string.clj:191)
clojure.string/join (string.clj:180)
shadow.build.targets.browser/flush-unoptimized-module-eval (browser.clj:522)
shadow.build.targets.browser/flush-unoptimized-module-eval (browser.clj:506)
shadow.build.targets.browser/flush-unoptimized-module (browser.clj:661)
shadow.build.targets.browser/flush-unoptimized-module (browser.clj:656)
shadow.build.targets.browser/flush-unoptimized/fn--56013/fn--56015 (browser.clj:683)
clojure.lang.PersistentVector.reduce (PersistentVector.java:343)#2024-01-2408:55weiupdate: found the culprit but not sure what's wrong with it. the OOM goes away when I comment this out. https://gist.github.com/yayitswei/364f7106d606de27d57fb27c63cc80f2#2024-01-2410:24Geoffrey GaillardNot clear what’s happening. But I have an intuition.
Could you please try adding :loader-mode :default under shadow-cljs.edn -> :builds -> :your-build-id -> :devtools. Like this:
{:builds
{:dev
{:target :browser
:devtools {...
:loader-mode :default
...}
...}}}#2024-01-2411:19Dustin Getzthe “IC branch” is very stale, please upgrade to the maven release#2024-01-2418:39weiah sorry i meant i was on 4b543d46 when IC got merged#2024-01-2418:41wei@U2DART3HA that fixed it, thanks! i see now that electric-fiddle has that line, but i'm migrating an older electric-starter-app project. so :loader-mode :default is not the default? 😅#2024-01-2418:43Dustin Getzwhat does it do? I don't see it in the shadow docs https://shadow-cljs.github.io/docs/UsersGuide.html#2024-01-2421:16Geoffrey GaillardIt's not documented.
Shadow can perform hot code reload in two ways:
• :eval (the default) will send compiled js code to the browser through the shadow websocket and call js/eval on it. So it sends all the code as a single string. It's fast for small programs.
• :default (not the default) is the legacy method. It loads one js script per compiled namespaces. It is slower for small programs because it performs a network request per compiled namespace. It is faster for larger programs because it doesn't concatenate tons of code into a single string. #2024-01-2421:17Dustin Getzwe depend on this? we should talk to thomas#2024-01-2512:52Dustin Getz@U066TMAKS please upgrade to the maven release and let us know if the issue persists, I spoke with Geoffrey and we don't think the :loader-mode :default flag is needed after IC, though we are not absolutely certain#2024-01-2514:33weiwill report back when i upgrade, been putting it off due to the breaking websocket change#2024-01-2609:23weiupdate: latest master b32ac seems to work without :loader-mode :default for my app#2024-01-2416:36braai engineerWill tomorrow’s talk be recorded or have a livestream for remote viewers? I want to attend remotely, but don’t see any mention on the event: https://www.meetup.com/clojure-nyc/events/298312289/#2024-01-2417:03Dustin Getzphysical event in NYC, no stream#2024-01-2417:03Dustin Getzcontent will be streamed in upcoming months, this is just a dress rehearsal basically#2024-01-2419:06Vincentwho bought bitcoin in 2013 and wants to send us all to NYC for the weekend? x)#2024-01-2419:06Vincentwho bought bitcoin in 2013 and wants to send us all to NYC for the weekend? x)#2024-01-2423:59VincentHi everyone, I decided to switch to datomic and I am wondering about clj -A:dev:datomic-browser do I need to run the datomic agent for this to work and is there a specific one?#2024-01-2501:20Dustin Getzafter my NYC talk is done i will take a look at the electric fiddle demos#2024-01-2617:30Geoffrey GaillardWe’ve added a readme for https://github.com/hyperfiddle/electric-fiddle/tree/main/src/datomic_browser
It explains how to set datomic up and how to run it for the demo.#2024-01-2510:03avocadeMaybe @xifi or @ggaillard can have a go at this one: we're getting "white screen of death" (aka wsod) quite often during normal usage, where suddenly the app shows a blank white page when we go back to the tab after some time (sometimes half an hour, sometimes hours, or overnight). The console tends to show WebSocket is already in CLOSING or CLOSED state, or Websocket error: Network process crashed.
A curious thing is that the blank-page syndrome seems to only affect Chrome (not Safari or Firefox – all latest versions and all on macOS). But I got a similar issue in Safari yesterday where it still showed the app UI, but it was totally unresponsive (websocket failure according to the logs). It's not just showing a blank page to the customers so at least it looks less bad 🙂 But they still would have to reload the page to get the app usable again.
Appreciative of any ideas if you've seen/dealt with this before 🙏#2024-01-2510:23avocadeAnecdotal stuff from another app than ours is that the following page also gives me wsod if I leave it alone in the bg for some time, but this time in Safari:
https://dustingetz.electricfiddle.net/electric-fiddle.essay!%45ssay/electric-y-combinator
It's instead a Reactor failure: – Yn. I guess it's old code, but at least it's a data point for giving blank screens unexpectedly.#2024-01-2517:02VincentWSOD o.o#2024-01-2519:06Geoffrey GaillardThank you for the report. We managed to reproduce a similar behavior. We will give it a look.#2024-01-2519:34avocadeThanks @ggaillard, I'm continuing to track these issues and see where I can dive in further to gather more data.#2024-01-2616:26Dustin Getzthe safari issue may be different than the chrome issue#2024-01-2616:28Dustin Getzthe chrome issue i think we understand, if i am not mistaken we have dealt with this chrome issue in the past and merely regressed something#2024-01-2616:42Geoffrey GaillardPushed a fix to master. The Electric client will now try to reconnect when the tab is brought to the front.
Technical explanation:
• the https://dustingetz.electricfiddle.net/electric-fiddle.essay!%45ssay/electric-y-combinator runs on an old Electric version.
• There seem to be a bug where the server crashes after some idle time.
• The server kills the socket and the client doesn’t try to reconnect.
I changed the Electric client reconnection strategy. Electric will now always try to reconnect, but only if the tab or window is visible to the user. (https://github.com/hyperfiddle/electric/commit/54b423cd0c88cbee6a2c9053f5912ee2c5ef7832)
@U09K620SG I don’t think this is worth a changelog entry.#2024-01-2616:44Dustin Getzso iiuc there is a new chrome only bug where the server crashes when idle AND chrome is on the other side?#2024-01-2616:54Geoffrey GaillardNo, sorry for the confusion.
1. There was a bug in the past. I couldn’t reproduce the issue on latest master.
2. There was logic in electric_client.cljs saying: "Only reconnect on your own if we know why the connection failed."
I changed 2. to be "always reconnect (with exponential backoff) if the tab/window is visible to the user. If it’s not, wait for it to become visible"#2024-01-2617:12Dustin Getzit sounds like we should log a ticket to understand why the connection is failing in an unclean way?#2024-01-2617:21Geoffrey GaillardSure#2024-01-3018:05avocade@ggaillard we have tried the new branch since yesterday, and unfortunately we still see a wsod frequently. The error seems slightly different now though:
The connection to was interrupted while the page was loading
I'm seeing some exception in the logs on the backend, both locally and deployed, so it might be that which is the reason for the app failing. But it seems the frontend should still be able to auto-refresh with the new change, perhaps not. Not easy to debug this 😕#2024-01-3019:41Geoffrey GaillardWe found the origin of this error.
When one’s computer goes to sleep and is then woken up, the browser will send a websocket ping frame to the server to check if the socket is still open.
If it doesn’t receive a pong after some hard-coded time, it will consider the connection to be broken and show the above message.
Proof: we captured and inspected TCP frames with wireshark.
Turns out we forgot to answer to pings in our WS server implementation. We assumed by mistake pongs were sent automatically (httpkit does that, not jetty)
We’ll work on a fix.#2024-01-3020:02avocadeVery interesting! I was just going to post another data point that suggests you fixed wsod for this app I mentioned before:
https://dustingetz.electricfiddle.net/electric-fiddle.essay!%45ssay/electric-y-combinator
I'm getting the same error as we do, The connection to was interrupted while the page was loading., but I'm not getting a WSOD and this tab has been open for a good while now 👏:skin-tone-2: So this app seems to reload correctly. Which leads me to believe there's something in our app that malfunctions, eg a reference that times out and disappears or something else reference/memory-related (due to it taking some time to occur).
Just shortly re this:
> When one’s computer goes to sleep and is then woken up …
Our WSODs happen even though the computer is on all the time. My laptop never sleeps while connected to power in my office, only when I travel, so it's been on for days now (sure the screensaver has been going on and the display sleeps but not the computer), and the wsod still appears sometimes after less than an hour. (Also we're using httpkit so maybe the pong were actually sent for us then, not sure.)
But it will be very interesting to see if your upcoming fix allows our app to refresh correctly even if it crashes on the backend. Again thanks for all your help, this would be great to get to the bottom of 🙌:skin-tone-2:#2024-01-3113:18avocade@ggaillard another anecdote: when I run locally and terminate the backend, the app immediately goes to WSOD with WebSocket connection to '' failed:.
A thought: would maybe be preferable if the app stays visible at all times, even if the backend connection is severed, so it never goes to just a white screen. At least it makes the app not look totally broken in an unexpected way. Of course when the user tries to interact it will be unresponsive, but at least the UI doesn't disappear on them 🙂#2024-02-0110:36avocade@ggaillard I updated to your latest commit on master yesterday afternoon and deployed, have tested overnight and we still get wsod. this error message now:#2024-02-0110:54Geoffrey GaillardThis could be caused by many low-level network issues. Can you reproduce this reliably? Please add timestamps to your screenshots.#2024-02-0115:30avocadeYeah it can be a lot of things causing the network to go down. But it seems the refresh-functionality should resume it anyway regardless. If the backend is down due to a deploy, then it's up again within minutes. And I'm getting this locally as well when the backend is constantly running, but when the tab (or just the app often), is just in the background for some time.
I wish I could get timestamps for those error logs, but the console doesn't give them in many cases. when we use timbre internally we get timestamps which is very helpful.
We'll do some more digging and see if we can come up with other things to try and triage the problem 👍#2024-02-0116:54Geoffrey GaillardI agree it should reconnect automatically anyway.
To display timestamps:#2024-02-0118:35avocadeWeird, thought I had that turned on everywhere 🥸 Thx for the pointer.#2024-02-0203:45VincentPhilosophical thought: Is there some way we can figure out what layer of the protocol stack* we are [with]in ? electric invokes debugging of all layers simultaneously x)
Also is protocol stack the term?
System Stack
Total Stack
Multilayer Stack#2024-02-0210:02avocade@ggaillard interesting result overnight, with latest electric master, deployed to two different servers (computer was on all night but "display sleeped" after 30 minutes, and the apps were in the background). all tabs got WSOD:d.
Different results in different browsers:
• Chrome: no error message at all, just a warning after ~1h20m: electric_client.cljs:54 [Violation] 'close' handler took 1094ms (see screenshot)
• Safari: error message after ~1h: WebSocket connection to '' failed: The operation couldn’t be completed. Connection reset by peer
• Firefox: error after ~1h: The connection to was interrupted while the page was loading.#2024-02-0210:26Geoffrey GaillardThis is helpful. Thank you. I’ll look into it.#2024-02-0210:42avocade🙏#2024-02-0509:13Geoffrey GaillardHere is my current understanding of the issue: sometimes while the tab is in the background or the screen is sleeping, either :
• a deployment will cause the electric client to disconnect, reconnect, detect a client/server version mismatch and trigger a page reload
• or the authentication token will expire, and trigger a redirect and/or a page reload.
Because the tab is considered to be in the background, it got limited resources.
Your JS bundle is large and the browser might fail the download the new file (process throttled or some exceeded quota - not sure).
Therefore the page doesn’t load completely, and you get a white page.
Screenshots of what happened in background, respectively:
1. refresh due to a deployment
2. couldn’t load main.js after auth redirect#2024-02-0509:50avocade@ggaillard interesting point, so it could be a more prominent issue for us since we end up with a bigger JS bundle than most/all (?) other apps that use electric…
But I've also seen the wsod locally within a short amount of time (<30 minutes) where the app was running the whole time. And also on prod (http://app.multiply.co) where we only deploy a few times per day. Those wsod:s happened way faster than auth0 token expiration (which is a few hours at least), and no deploys happened during that time.#2024-02-0509:50avocadeAnother thing is that I have started getting wsod:s for this app again, which I just assumed you had updated to the latest version (with the refresh-fix). but now I'm thinking that I was wrong about that 🙂 I thought it was done since I didn't get a wsod for a few days or so, but then on Friday I saw it again:
https://dustingetz.electricfiddle.net/electric-fiddle.essay!%45ssay/electric-y-combinator#2024-02-0610:01Geoffrey GaillardHi Oskar
• From what we can reproduce, every time the ws connection drops for any reason, we observe it reconnects successfully.
◦ We tested with chrome, firefox, locally and on your prod and staging env.
• We’ve noticed some WSOD but they don’t seem to be related to electric (e.g. screenshot)
◦ We don’t know why a JS file is failing to load 22 min later, but this is not related to electric.
#2024-02-0610:16Geoffrey Gaillard#2024-02-0615:45avocadeThat’s interesting, I haven’t seen a Reactor failure with our app for many weeks now. Thanks for the screenshot 🙏:skin-tone-2:
PS. We’re currently removing a big chunk of the app (old now unused parts), so will be interesting to see if the JS becomes substantially smaller in size, and if so helps with the WSODs. #2024-02-0616:52Geoffrey GaillardI don’t trust my claim about the js file size anymore. Your staging env has a large js file (for debug reasons) but the prod js file is of an acceptable size. So it seems the js file size is not an issue here.#2024-02-0617:35avocadeThanks for the clarification.#2024-02-0807:38henrikI don’t know if this ever were in question or not, but I recently inserted a useless div at the top, as the very first thing happening in the app (after binding node) in order to see if at least that div was present when the app WSODs.
It’s not, so that should definitively exclude any chance of it being related to authentication, as our authentication happens entirely within Electric, and after mounting this div.#2024-02-0914:04henrikI switched to Jetty, to see if this was Http-Kit related. Jetty still exhibits the same issue, but gives this server side:
2024-02-09T12:54:02.329Z Clavain.local ERROR [hyperfiddle.electric-ring-adapter:?] - Websocket error
java.lang.VirtualThread.run VirtualThread.java: 311
java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run ThreadPerTaskExecutor.java: 314
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run SelectableChannelEndPoint.java: 53
org.eclipse.jetty.io.FillInterest.fillable FillInterest.java: 99
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded AbstractConnection.java: 322
org.eclipse.jetty.websocket.core.WebSocketConnection.onFillable WebSocketConnection.java: 342
org.eclipse.jetty.websocket.core.WebSocketConnection.fillAndParse WebSocketConnection.java: 484
org.eclipse.jetty.websocket.core.WebSocketCoreSession.onEof WebSocketCoreSession.java: 229
org.eclipse.jetty.websocket.core.internal.WebSocketSessionState.onEof WebSocketSessionState.java: 168
java.nio.channels.ClosedChannelException:#2024-02-0914:05henrikThis is Jetty 12 using info.sunng/ring-jetty9-adapter#2024-02-0914:15Geoffrey GaillardRecent electric doesn’t need ring-jetty9-adapter anymore. Are you running an old version of electric?#2024-02-0914:16henrikNo, I just wanted virtual threads support, which is dicey in Jetty 11.#2024-02-0914:17henrikring-jetty9-adapter has a misleading name, the actual version of Jetty depends on the version of the lib.#2024-02-0914:18henrikThe version of Electric is the latest master: f8776792bb24a3b8b81baa509e5758fea7d20d28#2024-02-0914:26henrikIt’s configured like this:
(jetty/run-jetty routes
{:host ip
:port port
:join? false
:virtual-threads? true
:ws-max-idle-time 60000
:ws-max-text-message-size (* 100 1024 1024)
:configurator (fn [server]
(add-gzip-handler! server))})
Which should be equivalent to the servlet stuff from starter/fiddle. Jetty 12 doesn’t have a servlet API anymore. Edit: Actually, modulo binary size, which isn’t exposed.#2024-02-0914:59avocadeSame wsod-issue with jetty then… this becomes stranger and stranger. #2024-02-1507:35henrikWith 0cfb98429f0e9b4cb7822173d55d1498ce2dd972, WSOD remains, but no errors show up in the server-side logs anymore.
Client side, these things show up around the time when it should have WSOD’d:
main.js:4410 [Violation] 'message' handler took 304ms
main.js:7270 [Violation] 'setTimeout' handler took 242ms
main.js:7270 [Violation] 'setTimeout' handler took 70ms
main.js:7595 [Violation] 'close' handler took 549ms
WSOD could be tied to app size. Anecdotally, we’re getting more complaints of it (app size has grown ~100kb). But unfortunately, I don’t have a more precise measurement than that.#2024-02-2011:11avocade@ggaillard I think I've conclusively refuted the hypothesis that it's about the app bundle size. In our boot sequence, if we stop at an earlier point before loading the full app (we have a Welcome screen if the user is not a member of any workspaces yet, like on first signup), then the auto-reload works fine and it doesn't get stuck indefinitely at the white screen like we get when the main app UI is fully loaded.
If the app bundle size (or related) would be a cause of this, then shouldn't it always fail regardless of how much of the app (and data) is loaded?
This is confirmed on my M1 Max laptop in all three browsers (chrome firefox safari), and also on an iPhone 15 Pro. On the iPhone it's very easy to get the app to "unload" (since it seems to eat a lot of RAM) – just opening the Camera app usually does it. When I then go back and view the tab where it stayed on the Welcome-screen it reloads exactly as it should and everything is fine; but when I view the tab where the main app is loaded, it's white and unresponsive (even the "Refresh" button has disappeared from the address bar I noticed, but that's just anecdotal and I don't have insight into the internals of Apple's Safari team to know if this is significant 😅).
I've been and am currently attempting to sort of "bisect" to see where in the chain it starts just giving up and dying on a blank screen forever. Could be related to RAM possibly, could be the amount of data loaded, amount of listeners/missionary/reactive stuff in-flight, no clear idea at this point. But if it's just RAM, then after the browser kills the tab in the bg (as normal) it should not prevent the auto-reload when I view the tab again. So it feels like the app/electric is getting itself into a corner it can't back out of and then just dies.#2024-02-2011:19henrikBased on this, it might be something related not to the bundle size, but the payload size transmitted? Alternatively, the “depth” of the stack of reactive functions? That is, given that a simpler screen in the same app (that omits most of the app structure) auto reloads fine, while the full app does not.#2024-02-2014:24avocadeAnother question @ggaillard: since we don’t see any output in the browser console logs when WSOD happens, is there possibly some “insane mode” for logging that we could enable in electric? That would give us everything? 😅 #2024-02-2016:00Geoffrey GaillardThe electric client already log everything meaningful. It will log about everything but received and sent messages, meaning:
• connection attempts
• disconnects
• reconnects
• exceptions.
You can also look at your devtools "Network" tab to see websocket messages and control frames (screenshot on firefox).
If there are no logs, then the electric client is not running.
I tried to reproduce the issue on my phone as you described:
• navigate to [Violation] 'close' handler took 410ms electric_client.cljs:54 warning, while the right hand side does not.
Is it possible that the app takes so long to tear down that it disrupts the reconnector?#2024-02-2017:09avocadeMm we've seen those [Violation] warnings for a long time now. Could it perhaps "time out" trying to reconnect…#2024-02-2115:21henrikIt looks like we’ve eliminated WSOD for now.
I went in and started to comment out large parts of the app to see where it would start working. Commenting out stuff didn’t make any difference.
But I had built a boot helper used early on in the app to be able to create reactive functions that kind of worked as loosely coupled middleware. Replacing that with tightly coupled function calls made the problem disappear.
I’ve put the code that I removed in an Electric starter here: https://github.com/fluent-development/electric-boot/blob/boot/src/electric_starter_app/main.cljc
Note, however, that this works fine in Electric starter. So we’re rid of the problem, but don’t understand why it worked. We’ll be monitoring the situation.
The other weird thing is that I built it while investigating WSOD. :man-shrugging:
Thank you very much for the help @ggaillard, all is good for now.#2024-02-2118:49avocadeThanks for all your help @ggaillard, this was as weird one and you've been most helpful in this process 🙏 Hope it stays fixed now 🥂#2024-01-2606:04weiseeing some unexpected behavior, am I missing something obvious??
(ns app.test
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom :refer [div text]]))
#?(:clj (def !a (atom "a")))
(e/def a (e/server (e/watch !a)))
#?(:clj (def !b (atom "b")))
(e/def b (e/server (e/watch !b)))
#?(:clj (def !c (atom "c")))
(e/def c (e/server (e/watch !c)))
(e/defn Test []
(e/client
(div
(div (text "a: " (pr-str a)))
(div (text "b: " (pr-str b)))
(div (text "c: " (pr-str c))))
(println a)
(println b)
(println c)))
this is on latest master (`b32ac9`)#2024-01-2606:18weiwell this works as expected in electric-fiddle so it must be something screwy in my app. any tips on locating the issue?#2024-01-2606:18wei#2024-01-2607:00weion my original project, I tried pasting in the Toggle demo code (https://electric-examples-app.fly.dev/(electric-tutorial.demo-toggle!%54oggle)/) and got this error:
22:57:17.316 ERROR [qtp362743760-80] hyperfiddle.electric - #error {
:cause check failed: (some? (get (aget frame frame-slot-dynamic) symb)) for nil
:data {:hyperfiddle.electric.debug/trace [{: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [/test2 <exception>], :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [#object[HTMLBodyElement [object HTMLBodyElement]] <exception>], :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/args [], :hyperfiddle.electric.debug/serializable true}], :hyperfiddle.electric/type :hyperfiddle.electric.debug/trace}
:via
[{:type hyperfiddle.electric.FailureInfo
:message check failed: (some? (get (aget frame frame-slot-dynamic) symb)) for nil
:data {:hyperfiddle.electric.debug/trace [{: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [/test2 <exception>], :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [#object[HTMLBodyElement [object HTMLBodyElement]] <exception>], :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/origin #uuid "af9a7076-d898-4b7f-9acb-4aa19cbada54", :hyperfiddle.electric.debug/args [], :hyperfiddle.electric.debug/serializable true}], :hyperfiddle.electric/type :hyperfiddle.electric.debug/trace}}]
:trace
[]}
in case default branch
in (case 0 ...)
in reactive (defn Main nil ...) line 19
in (try ...)
client logged an exception, too #2024-01-2607:24weithink I found a minimal repro. seems you can't import watches
(ns app.teststate
(:require [hyperfiddle.electric :as e]))
#?(:clj (def !a (atom "a")))
(e/def a (e/server (e/watch !a)))
;---
(ns app.test
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom :refer [div text]]
#?(:cljs [app.teststate :refer [a]])))
(e/defn Test [] (e/client (div (text "a: " (pr-str a)))))#2024-01-2607:52xificurCthanks for the report and minimized repro! The last snippet, you tried it in electric-fiddle?#2024-01-2608:18xificurCI did not reproduce locally. Comparing the code you posted to what I typed, the #?(:cljs conditional on the require is likely the culprit. If a is an electric def the server needs to know that too#2024-01-2608:21xificurCalthough adding that locally didn't repro either. Might be a stale tab too#2024-01-2609:09weii see, makes sense that both client and server need to know about the e/def. thanks for following up. as written (with the conditional) it seems to mess up EVERYTHING even outside of that ns. easy for a newbie like me to get confused#2024-01-2609:21weiconfirmed that it works without the conditional ✅#2024-01-2615:21Dustin GetzTo summarize, we've learned that #?(:cljs ...) around a namespace that contains electric expressions that have server contents, causes a client/server mismatch under IC. Furthermore, the mismatch is not detected, is allowed to run and we see spooky undefined behavior due to the programs being out of sync. Correct?#2024-01-2615:54weiYes, great summary!#2024-01-2613:48telekidThanks for the awesome talk at Clojure NYC yesterday! I learned a ton. Great to hear about the motivation behind differential.#2024-01-2614:36avocadeVideo or it didn't happen.#2024-01-2615:17xificurCthe latter then#2024-01-3112:43avocadeI have a feeling a video'll turn up eventually. Looking forward to it :star-struck:#2024-01-2902:41chromalchemyI’ve successfully deployed to http://fly.io before. I thought I remembered a comment to maybe avoid it if you don’t need the distribution model?
I have a Digital Ocean droplet ready to go with a domain.. or I could use something else. Does anyone have any guidance on the simplest path? Is it easy enough to use an Uberjar directly? (the droplet has been previously provisioned with Biff setup bash script). So far, the docs are quite terse, and googling around uberjar stuff is a bit mystifying… . Would it be easier to use docker and Digital Ocean App Platform? Or just use Fly as before?#2024-01-2912:04Dustin Getzyes use uberjar#2024-01-2912:05Dustin Getzssh to droplet, run
java -cp target/app.jar clojure.main -m prod
#2024-01-2914:48chromalchemyThanks, will give it a go.#2024-01-2923:32chromalchemy@U09K620SG I got the XTDB Fiddle deployed to a Digital Ocean VPS droplet! 🙌clojure-spin
Took me all day grinding.
First I pulled Electric fiddle into my project on a new branch.
I got the fiddle running locally, learning to respect adding the prescribed XTDB_ENABLE_BYTEUTILS_SHA1=true
Then focusing on deployment:
I Complied the uberjar from the fiddle with the above command.
Ubuntu on my droplet was complaining about being old and unsupported, but wouldn’t update. So I wiped the droplet and started from scratch, and had to fiddle with ssh keys.
I wasn’t sure how much the system need to run a jar. So I installed just java and tried to run the jar at root. Nothing.
So I walked though the Biff https://github.com/jacobobryant/biff/blob/dev/example/server-setup.sh script, running the parts that looked meaningful to a clean build. I left out nginx thinking that would be handled by the jar.. Still nothing. So I set up nginx and a certbot, then it would load, but not showing any content. I needed to add the env var in the right spot on the jar call XTDB_ENABLE_BYTEUTILS_SHA1=true java -cp app.jar clojure.main -m prod
Up and working!! Reactive multiplayer feels magical! ⚡
I did get one crash with client_loop: send disconnect: Broken pipe . My internet connection went out about the same time. Will set up systemd to auto-reboot..?
In retrospect I was in a bit over my head, and probably should have just gone with docker and a service to save time. But i did get some hard won learnings and confidence flexing on the devops side (a weak spot for me).
I’m glad to have the electric fiddle repo as a launchpad.
I was tempted to go with the starter repo, but I figured this was more likely to be up to date and provide more working example code. Thanks for the considerations on that.#2024-01-2923:51Dustin Getzare you saying our xtdb fiddle is missing a readme step or something? sorry you had to debug that, it doesn’t seem like electric issues, these all look like xtdb issues#2024-01-2923:53Dustin Getzxtdb fiddle - https://github.com/hyperfiddle/electric-fiddle/tree/main/src/xtdb_demo#2024-01-2923:53Dustin Getzit has the env var#2024-01-2923:54Dustin Getzfwiw the starter app is up to date as of friday or so, but we don’t intend to maintain database specific starters outside of the fiddle repo#2024-01-2923:57Dustin Getzoh i see, you didn’t know how to install clojure cli#2024-01-3000:02chromalchemyI think the fiddle, and the documentation is fine. The environmental var was in the documentation. That was all about me noodling through the deployment process outside of electrics concerns. #2024-01-3000:04chromalchemyI have the closure CLI command installed locally. I’m not sure if I installed it remotely. I was under the impression and I didn’t need that to run the Uber jar.#2024-01-3000:05chromalchemyJust a little journal about me learning to deploy an Uber jar. #2024-01-3000:08chromalchemyThere is a little variation on where the environment variable goes in the clojure CLI, versus the Java commands. Thankfully, I got a error read out in the consul, when it wasn’t included properly, when I tried to run the jar in production. And that alerted me to get it right#2024-01-3000:26Dustin Getzyou do not need clj for uberjar, sorry#2024-01-2909:24AvivHey everyone,
Does datomic-browser work with datomic-cloud?
Thanks 🙏#2024-01-3008:27Geoffrey GaillardThe datomic browser uses the datomic peer api. We haven’t tried to adapt it to datomic cloud.#2024-01-3008:48AvivOh ok, thanks!
are there any plans to support it?#2024-01-3009:06Geoffrey GaillardI’m not sure, I’ll ask#2024-01-3014:23Geoffrey GaillardWe are not planning to adapt datomic-browser to datomic cloud. But if you want to do so, there is datomic-cloud compatible code in our https://github.com/hyperfiddle/electric/tree/master/src/contrib.#2024-01-3015:10Avivthanks! i’ll take a look#2024-01-2922:09VincentSo... using the new fiddles approach...#2024-01-2922:13VincentI have "app navigation" where I have it look at the first subfolder in the url ... right? localhost:8080/browse for example#2024-01-2922:17VincentHowever, the hyperfiddle examples application overrides this on page refresh....#2024-01-2922:17Vincentwants things to conform to (ns electric-fiddle.main
(:require
[hyperfiddle :as hf]
[hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.router :as r]
[electric-fiddle.index :refer [Index]]
))
(e/defn NotFoundPage [& args] (e/client (dom/h1 (dom/text "Page not found: " (pr-str r/route)))))
(e/defn Main [ring-req]
(e/server
(binding [e/http-request ring-req]
(e/client
(binding [dom/node js/document.body]
(r/router (r/HTML5-History.)
(dom/pre (dom/text (pr-str r/route)))
(let [route (or (ffirst r/route) `(Index)) ; route looks like {(f args) nil} or nil
[f & args] route]
(set! (.-title js/document) (str (some-> f name (str " – ")) "Electric Fiddle"))
(case f
`Index (Index.)
(r/focus [route]
(e/apply (get hf/pages f NotFoundPage) args))))))))))
#2024-01-2922:45VincentAnd I'm wondering what I ought to do to.... take this over elegantly 😅#2024-01-2922:49VincentAlso, the "not found page" is optimistic and never evaluated, I think. Because it has to conform to the very specific formula for what a page is, otherwise it's not ISeqable and the "page not found" page is actually a name and the word nil so... just something to keep in mind. Wrenching out this plumbing is taking some effort )#2024-01-2922:58VincentNVM noticing a new electric-starter-app#2024-01-2923:46Dustin Getzyes use starter app#2024-01-3008:25Geoffrey GaillardAre you saying there is a conflict between the default client side routing and your custom server side routing?#2024-01-3012:02VincentEverything I said is irrelevant )
Was gutting the demos app ... and there is no need for that now#2024-01-3008:21joshchofiddles are quite nice. thanks team!#2024-01-3008:48Eric DvorsakHello!
I've been trying to run this gist https://gist.github.com/dustingetz/1960436eb4044f65ddfcfce3ee0641b7
The second snippet with the !dirty gets me unbound electric var electric-starter-app.main/!dirty`, did I somehow manage to break something is so few lines or is the snippet not working? If so what is happening exactly? does binding not "see" previous bindings in the same form?#2024-01-3008:52Geoffrey GaillardIt looks like there’s a mistake in the snippet. Sorry.
binding binds in parallel (in both Electric and Clojure).
Until we revise the snippet, you can try with:
(binding [!dirty (atom 0)]
(binding [dirty (e/watch !dirty)]
...)#2024-01-3008:57Eric Dvorsakok thanks!#2024-01-3008:57Eric DvorsakDo you still use something like mount with electric to manage eg database connections that need to be closed?#2024-01-3009:03Geoffrey GaillardNo need, Electric has the mount/unmount lifecycle baked in.
(e/defn GetDbConn []
(let [my-closable-conn (get-db)]
(e/on-unmount #(.close my-closable-conn))
my-closable-conn))#2024-01-3009:46Eric DvorsakI get
ERROR hyperfiddle.electric: #error {
:cause Wrong number of args (0) passed to: hyperfiddle.electric.impl.runtime/constant/fn--7241
...
in dynamically bound ( electric-starter-app.main/GetDbConn )
In a binding like this:
(binding [pg-conn (GetDbConn)] ... )
#2024-01-3009:50Geoffrey GaillardElectric functions (`e/defn` and e/fn) are called with new: (new GetDbConn) or the equivalent (GetDbConn.).#2024-01-3010:29Eric DvorsakAh thanks!
but how does it work for e/fn then, eg in that snippet:
(CodeMirror. {:parent dom/node}
(e/fn [x]
(try (edn/read-string x)
(e/server (reset! !sql-map x))
(catch #?(:clj Throwable :cljs :default) t
nil))
x)
contrib.str/pprint-str
(e/server sql-map))
I suppose that is why I get "#object[TypeError TypeError: n.call is not a function]"#2024-01-3010:36Geoffrey GaillardI don’t see anything wrong with this snippet.
e/fn is called like e/defn:
(let [Identity (e/fn [x] x)]
(new Identity 1))
or shorter: (new (e/fn [x] x) 1)#2024-01-3010:40Eric DvorsakI think what is wrong is that the code mirror expects a clojure/script write-fn, but then you can't have e/server in it?#2024-01-3011:10xificurCa cljs fn and an electric fn are 2 distinct objects. If you are interoperating with a js library that expects a (cl)js fn you cannot pass it an electric fn.
Are you trying to use our contrib codemirror namespace?#2024-01-3011:26Eric Dvorsak@U09FL65DK yes#2024-01-3011:35Eric DvorsakI'm trying to update the value on the server when the content changes#2024-01-3012:01VincentMy guess is that the try catch must be fully insulated on one side or the other#2024-01-3013:00Eric Dvorsakevent just that fails:
(CodeMirror. {:parent dom/node}
(e/fn [x]
x)
identity
@!input)
#object[TypeError TypeError: n.call is not a function]
```#2024-01-3019:56VincentWhat is CodeMirror and what do you want to achieve#2024-01-3019:57Eric Dvorsakit's from the contribs: [contrib.electric-codemirror :as codemirror :refer [CodeMirror]]#2024-01-3019:58Eric DvorsakI'm trying to update the value in the server when the content changes#2024-01-3109:20Eric Dvorsak(e/defn GetDbConn []
(let [my-closable-conn (get-db)]
(e/on-unmount #(.close my-closable-conn))
my-closable-conn))
wouldn't that open/close the connection for every client? if I'm using a connection pool I wouldn't want that?#2024-01-3109:41Geoffrey GaillardDepends on what get-db does. If it acquires the connection from a pool or if it builds a fresh one.#2024-01-3109:43Eric Dvorsakyeah in my case it creates the connection pool:
(jdbc.connection/->pool
HikariDataSource
{:minimum-idle 10
:maximum-pool-size 10
:username "user"
:password "password"
:driverClassName "org.postgresql.Driver"
:jdbcUrl "jdbc:"})#2024-01-3009:16joshchofor people who have experimented with rich text editors in electric, which libraries did you use?#2024-01-3013:48weii've gotten vanilla lexical to work with a lot of hacking. maybe if i used react lexical the integration would have been simpler.#2024-01-3105:45zeitsteinOff topic, but wondering if you've used ProseMirror and how you would compare it to Lexical @U066TMAKS? (Unrelated to Electric)#2024-01-3106:32weii haven't used prosemirror, but i think it's supposed to be the more mature option#2024-01-3011:58VincentI think it's so cool we can do this now
(e/defn Main [ring-request]
(e/server
(binding [e/http-request ring-request]
(e/client
(binding [dom/node js/document.body]
(ShompMain.))))))
#2024-01-3012:02joshchowhat is the purpose for exposing ring-request in Main? (not too familiar with ring)#2024-01-3012:04Vincentaccess to request headers and session variables#2024-01-3012:04joshchois there an example in fiddle?#2024-01-3012:04joshchoif not, would be very nice#2024-01-3012:11Vincenti figured out how to do it from the extended chat demo, which may or may not exist somewhere currently#2024-01-3014:22Geoffrey GaillardYes extended chat demo reads cookies. It is in the electric-tutorial fiddle.#2024-01-3015:34joshchoty @U2DART3HA!#2024-01-3012:54HendrikI observe a strange behaviour with e/for-by iteration over a collection, where multiple dom nodes are created in each iteration step. When a item in the middle of the collection gets removed, then the ui gets cluttered. Is this behaviour expected? (i paste an example with screenshots in the comments)#2024-01-3012:54Hendrik(def !my-list (atom [1 2 3 4 5 6]))
(e/defn Entry [text]
(e/client
(e/on-unmount #(println "unmount" text))
;no single containing dom element
(dom/div
(dom/text text))
(dom/div
(dom/text "Foo"))))
(e/defn List []
(e/server
(let [my-list (e/watch !my-list)]
(e/for-by identity [v my-list]
(e/client
(Entry. v))))))
(comment
(reset! !my-list [1 2 4 5 6]); remove from the middle
)
#2024-01-3012:55HendrikUi before removal:#2024-01-3012:55HendrikUi after removal:#2024-01-3013:52weii couldn't repro that, this is what i get using your snippet:#2024-01-3013:53weii recently had some strange behavior around conditionals causing client/server mismatches, maybe check for that? https://clojurians.slack.com/archives/C7Q9GSHFV/p1706249048037089#2024-01-3015:11HendrikThanks for that hint. I carefully checked, that I do conditional requiring of namespaces that contain electric definitions. But I still get the strange behaviour. btw I am using electric version “v2-alpha-540-ga4699532”. Did you try to reproduce with the same?#2024-01-3015:16weiit's the conditionals that cause the undefined behavior. e/defs need to be defined on both the server and the client (no conditionals) and there's currently no warning if one side is missing#2024-01-3015:16weii'm on the latest master#2024-01-3015:20HendrikI have no e/defs in conditionals.#2024-01-3015:20HendrikI try latest master later this day#2024-01-3015:31HendrikI stil get the error with latest master. I only get corret behaviour, if I wrap each iteration step in another div like so
(dom/div (Entry. v))
#2024-01-3016:22weihow about try to repro on a clean clone of electric-starter-app or electric-fiddle? that would tell you if it's an issue with your specific app#2024-01-3021:26HendrikI testet with the electric fiddle repo and extended the hello fiddle:
#?(:clj (def !my-list (atom [1 2 3 4 5 6])))
(e/defn Entry [text]
(e/client
(e/on-unmount #(println "unmount" text))
;no single containing dom element
(dom/div
(dom/text text))
(dom/div
(dom/text "Foo"))))
(e/defn List2 []
(e/client
(dom/div
(e/server
(let [my-list (e/watch !my-list)]
(e/for-by identity [v my-list]
(e/client
(Entry. v))))))))
(comment
(reset! !my-list [1 2 4 5 6]); remove from the middle
)
(e/defn Hello []
(e/client
(List2.)
(dom/h1 (dom/text "Hello world"))))
I still get the wrong behaviour (see screenshot). @U09K620SG do you have any idea what could cause this behaviour?#2024-01-3021:31Dustin Getzwe will try to reproduce#2024-01-3021:32Dustin GetzI also suspect (like Wei) some sort of desync. The recent Electric incremental compilation changes made it easier to desync the frontend and backend using things like reader conditionals to cause the client and server to see different programs, we're still gathering info#2024-01-3021:34Dustin GetzIt could also be a bug, the IC changeset was huge#2024-01-3014:04weii'm mounting/unmounting a table of ~100 form ~1-2s delay. any suggestions for making this more performant? I also tried commenting out the input elements (keeping the row divs) and while it's better, there's still some delay. what's the overhead for e/defn and should i be trying to inline things here?#2024-01-3014:25Dustin Getzyeah e/fn overhead is high in v2, and worse, each electric-dom2 container (div, tr, etc) has a hidden e/fn wrapping the children. Really new is what has the overhead and in electric-dom2, new is everywhere. This is a top dev priority, Electric v3 will be better due to resolving a bunch of technical debt (Spring release) and one consequence of Electric v3 is that all of the hidden e/fns inside dom containers are no longer needed, which might solve this issue outright#2024-01-3014:26Dustin GetzYes there are optimizations you can do in the meantime – i'll get back to you, i'll make a short list#2024-01-3014:43weithanks! i can also wait for v3 if the optimizations are v2-specific. targeting a late spring release for this particular project#2024-01-3118:51Dustin GetzCan you copy paste the key parts of your code into a gist? DM me if you want to keep it private#2024-01-3118:55weisure, give me a few min 🙏#2024-01-3118:55weibtw, does e/watch have significant overhead?#2024-01-3118:56weirelative to, say, instantiating a e/defn#2024-01-3118:57Dustin Getzum, i'm not sure. Both involve new#2024-01-3118:57Dustin Getzdo you have a lot of them?#2024-01-3118:59weimaybe a dozen? not as many as i have inputs and defns. i'll treat them the same in terms of cost then#2024-02-0113:51Dustin GetzTo follow up in public, step 1 is virtualized scroll, demo here: https://electric-datomic-viewer.fly.dev/(:app.datomic-browser!attribute,:abstract%52elease!name)#2024-02-0113:56Dustin GetzHowever, even in that demo, navigating between tabs is not instant, it's maybe 300ms to switch tabs (though not 1-2s). Some of this can be blamed on network delay but there is also clearly a waterfall load of sorts happening because the cells stagger in if you look closely#2024-02-0114:01Dustin Getzif the backend queries for the query/cells are slow, due to the waterfall this can really explode out, that's when you really need to optimize today. Heavy electric-dom2 usage also adds weight due to it's internal e/fns as mentioned above.#2024-01-3017:38Tommy JollyboatIs electric-starter-app intended to be maintained long-term as the clean starting point? I migrated to electric-fiddle a few weeks ago — now it seems the sensible thing is to move back. Another question -- is there any reason not to keep the remote and repeatedly rebase on top of it? I like the idea of automatically updating dependencies and so on.#2024-01-3017:50Dustin Getzhonestly we don't commit to anything, we're still triangulating what our requirements are and what user requirements are. electric-fiddle is intended to have "batteries included" and electric-starter-app is intended to document the cleanest possible entrypoint because electric-fiddle has extra complexity (due to so many apps co-existing each with its own classpath requirements)#2024-01-3017:52Dustin Getzthe reality is that your app http middlewares (auth and such) are customized in 100% of real apps, so you have no choice but to understand the 100 line example http entrypoint#2024-01-3017:55Tommy JollyboatOK, I'll stick with electric-fiddle for now, and yes I've been putting off understanding the underlying parts properly but I'll have to figure it all out one day!#2024-01-3017:56Dustin GetzI'll add that electric-fiddle is far too fiddley today, it was shipped quickly and we're working on ways to improve it#2024-01-3018:06Tommy JollyboatThat's true! I would probably get less conflicts on electric-starter-app. Thanks for the guidance as ever 🙂 Looking forward to the Differential talk#2024-01-3109:20Eric Dvorsak(e/defn GetDbConn []
(let [my-closable-conn (get-db)]
(e/on-unmount #(.close my-closable-conn))
my-closable-conn))
wouldn't that open/close the connection for every client? if I'm using a connection pool I wouldn't want that?#2024-01-3105:51VincentHaving used xtdb and datomic,#2024-01-3105:52VincentXTDB is amazing and uses under the hood a tool called Nippy to serialized native clojure datatypes giving you access to vector, set, all your favorites, as things you can write to DB in the value part of key, value#2024-01-3105:54VincentDatomic does not allow this, and for this restraint the rendering time and retrieval time seems far superior. Which is very interesting. For some save-to-db trade offs (Datomic is less convenient in some ways) you must figure out how to represent things in Datomic. For attributs/properties that can have many values you can simply have multilppe transactions and retrieve all of the things ever transacted. it lends into how you think about designing the data and the lookup.#2024-01-3105:55VincentDatomic is, in my opinion, the perfect solution for Electric applications. I am still getting everything to work just right, but the lookup speed is significantly better which will make a huge difference in live applications.#2024-01-3105:56VincentHowever, for the development phase, XTDB is amazing. Absolutely a joy to work with. And lets you fly schema-less. This is only partially true as when you write to db, you must conform to some degree of "everything you are looking for is there" otherwise you will get nubs that will not reveal themselves in queries of any limiting factors. Ok no problem.#2024-01-3105:56VincentSo I would recommend: if your app is very ambitious and you don't know the exact schema you need, consider using XTDB to start and develop it to a strong stable level#2024-01-3105:57VincentI spent some time writing all my "schema" over to a notebook (my sketchbook) and I found that we only have 7 data types across 2 different somewhat interwoven music apps.#2024-01-3105:57tatutwhile both have datalog, it isn’t a trivial change to go from XTDB to Datomic (or vice versa)#2024-01-3105:58VincentNot knowing this when I started, using XTDB made great logical senes. Now knowing the shapes rather clearly, codifying them into something more robust... Tatut brings up a great point.#2024-01-3105:58tatuttransacting is quite different, shredding into docs vs attributes#2024-01-3105:58VincentAh yes "docs" is the distinction... very good. Yeah it's not 1:1#2024-01-3106:00VincentBut, Tatut, you can, in theory, massage anything from xtdb shapes to a datomic shape )#2024-01-3106:00Vincenthowever, perhaps after a certain point of production being alive and ongoing, it would be not a fun transition.#2024-01-3106:01Vincentso I think early-on it is wise to consider what one needs, but yeah, very good point it's not as easy as drop in and go. it's a little more nuanced because datomic limits what goes in, more sacrosanct with more performance guarantees, xtdb a delight to work with / in as well.#2024-01-3106:02tatutI don’t think I’ve ever seen an actual “running in production” system change databases… there’s just so many devils in the details#2024-01-3106:03tatutunless it was architected from the ground up to support different databases, that is#2024-01-3106:06VincentXTDB and datomic are so nigh similar it is a tempting exercise#2024-01-3106:06Vincentmainly reforming collections into sequential transactns, i think#2024-01-3106:06Vincenti think#2024-01-3106:21tatutif you use the lucene index for full text search on XTDB, that has no equivalent afaict on Datomic… you would need something external for that#2024-01-3106:22Vincenttrue#2024-01-3106:24VincentIs there a way to get datomic to "react to changes' in electric? Not sure how to turn it into a live ref and not just a instance in time for db#2024-01-3109:11HendrikI noticed, that the shared snippets (see here https://clojurians.slack.com/archives/CL85MBPEF/p1685970532152309?thread_ts=1684166929.251079&cid=CL85MBPEF) of how to get a flow of datomic db changes, can’t be used with the reload wf(tools-namespace refresh etc.). If you use it out of the box, then you will notice, that some changes are ignored and then 2 to 3 transactions become visible at once. This is due to the fact, that each item in datomic’s transaction queue can only be consumed once. After the first namespace refresh /reload there are 2 consumers (the first one is dangling). To overcome this you must properly delete the queue consumer before refresh (add it to the stop command in mount, component or whatever state management library you use). It took me quite some time to figure this out. So I put it here, so that it may help others :)#2024-01-3112:52Dustin GetzFwiw, electric v3 + newest missionary m/signal + defonce solves this problem. The solution (m/signal + defonce) doesn't work in electric today because electric v2 still uses the legacy m/reactor and m/signal! (with a !)#2024-01-3113:03Dustin GetzLet me elaborate, the snippet you linked to actually doesn't contain m/signal! at all, because m/signal! is illegal at global scope (m/signal! is only valid inside the scope of a m/reactor). Missionary's new m/signal (no bang) resolves this (also removing m/reactor api entirely), but electric isn't upgraded to new missionary yet, that's part of electric v3#2024-01-3113:04Dustin Getzm/signal + defonce is the combination that guarantees a stable tx queue singleton instance#2024-01-3113:06HendrikThanks for your answer. I think that it is not possible with v3. I do not speak about hot code reloading, but tools-namespace refresh, which destroys all namespaces. So even defonce vars will be recreated after refresh.#2024-01-3113:11Dustin GetzDid you consider shutting down the Electric app on tools-namespace refresh?#2024-01-3113:28HendrikIf you mean with “shutting down”, that the jetty-server is restarted, then yes.#2024-01-3114:40Dustin GetzIIUC it should be sufficient after v3 to shutdown the app on refresh (which will dispose all managed resources including the global tx listener)#2024-01-3118:52grounded_sageWhat’s the news? electric-starter isnt a deprecated repo anymore?#2024-01-3118:52Dustin Getzit's been rebuilt#2024-01-3118:54grounded_sageOkay. What happened to fiddle being default? I was migrating and also building a new project on fiddle so curious what the status is. #2024-01-3118:54Dustin Getzelectric-fiddle is still a great place to start, we continue to move all our demos into it#2024-01-3118:56Dustin Getzif you're just fiddling around, fiddle is probably what you want. If you're making a real application, electric-starter-app has a simpler entrypoint for you to copy paste (though we still insist that you need to understand every line of your entrypoint, copy pasting is not a real solution)#2024-01-3118:58grounded_sageCool. Yea I’m quite familiar with electric now so not starting out. I’ll stick to the electric-starter. I thought fiddle was the primary/only future repo for starting and tracking changes.
I hadn’t dug in fully but also felt the entry points were simpler in starter. Thanks for the update. #2024-01-3118:57chromalchemyI deployed the xtdb-demo fiddle as is. After a while it crashes. (might be my environement or some xtdb thing). I haven’t tried to replicate failure locally yet.
Where do I look for logs? (with the fiddle setup).
What are the current memory requirements? My droplet has 1gb.#2024-01-3118:59Dustin Getzif you're ssh'ing into a linux box on digital ocean, i think capturing logs is not automatic, you have to write the bash code to capture logs#2024-01-3118:59Dustin GetzIt does sound like an OOM#2024-01-3119:01Dustin Getztry starting at 8gb and working your way down#2024-01-3119:03chromalchemyOk will try that first. Then… Biff has some logging setup I should be able to reuse.#2024-01-3119:04Dustin Getzjava ... >> logs.txt 2>&1 something like this#2024-01-3119:05chromalchemyThx#2024-01-3119:06chromalchemyWow, Droplet price goes up $40/mo for move to 8gb.#2024-01-3119:07chromalchemy#2024-01-3119:08chromalchemy~maybe that’s for a whole package though. will look for just memory.~
No couldnt find other options (to only resize memory)#2024-01-3119:31chromalchemyI bumped it to 2gb and will work up..
Is it your general opinion that the uberjar way is worth it for the simplicity? Maybe on a cloud deploy (http://fly.io or DO app platform) It would be cheaper for 8gb If I’m not using it continuously (interal app)#2024-01-3120:33Dustin Getzto be clear, electric does not use much memory, you are using an in process database#2024-01-3120:34Dustin Getzi dont know how much memory xtdb needs, but i do think $40 rounds to $0#2024-01-3120:35Dustin Getzcertainly is negligible compared to the debugging cost#2024-02-0100:49chromalchemyTrue. Just orienting towards reliable minimal deploy exprience I can use in other projects as well. I’m understanding XTDM is adding significant extra (memory, complexity) here.#2024-02-0101:59chromalchemy:face_with_peeking_eye: I believe my crashing was just my ssh session terminating, which was killing the process. Didn’t know about that. I just used tmux now to prevent this.#2024-02-0102:00Dustin Getzlol - you want ‘nohup’#2024-01-3121:23weihow much work would it take to get specter to work inside of an e/def? guessing a lot because of the macros and precompilation#2024-01-3121:25Dustin Getzi don’t know anything about spectre but i will point out that core.match works (or at least worked before the IC changeset, i don’t know if we re validated it yet)#2024-01-3123:43weinot a blocking issue for me, but just fyi I'm getting the following error trying to transform a watch:
(ns app.test
(:require [hyperfiddle.electric :as e]
[com.rpl.specter :as sp :refer-macros [select transform]]))
(def !my-list (atom [1 2 3 4 5 6]))
(e/def my-list (sp/transform [sp/ALL] inc (e/watch !my-list)))
{:error
{:cause "Cannot resolve 'var', maybe it's defined only on the client or needs to be referred in :require-macros."
:data {:form var, :in my-list, :context "server"}
:location "hyperfiddle.electric.impl.lang$fail_BANG_ at lang.clj:148" ...}}#2024-01-3123:48Dustin Getzwhile you’re there can you please capture the macroexpansion and paste here#2024-01-3123:49Dustin Getzthe spectre macroexpansion not the electric#2024-01-3123:56weiis this what you're looking for?
(macroexpand `(sp/transform [sp/ALL] inc (e/watch !my-list)))
=> (com.rpl.specter.impl/compiled-transform* (com.rpl.specter/path [com.rpl.specter/ALL]) clojure.core/inc (hyperfiddle.electric/watch app.test3/!my-list))
#2024-01-3123:57Dustin Getzha i guess, we’ll have to look more closely then to understand what’s wrong #2024-01-3123:58weiyes the expansion looks pretty mundane#2024-01-3123:58Dustin Getzyou can fallback to calling into a wrapper fn from electric and the wrapper uses spectre#2024-02-0100:03weithat works! 🙏#2024-02-0101:00chromalchemyThere are versions of specter fns that have different pre-compilation expectations. But probably still macro heavy.#2024-02-0108:51xificurCpath is a macro too, the expansion isn't complete. You can (e/def ^::lang/pprint-expansion .. to get the fully expanded code pretty printed in the REPL#2024-02-0101:06chromalchemyI appreciate that Electric is db agnostic. But does anyone have any soft guidance about what state persistance would be a recommended natural fit for electric atm. Like if heavier DB’s impose extra considerations or limitation of the reactive nature. I don’t have a strong sense of what my requirements even are. But is it maybe more solid to stick to using atoms until I definitely need something else, .. or is it all the same for electric… fine to just use those, or Datascript, Datalevin, or Datomic, up the (datalog) tree. (specifically thinking about Datascript vs XTDB vs Datalevin… or none of the above? do I need a real db?). I remember some warnings about needing to get the db connections stuff right with missionary, and that ideally the db would be built with electric. Just wondering about current perspectives on the state of electric vs dbs from a starter perspective. Thx#2024-02-0101:32Dustin Getzwe will be adding more datomic demos to electric fiddle and are quite happy with datomic onprem. which means it’s more likely that we already have any helpers you may need#2024-02-0102:03Garrett Hopper@U09K620SG, I've seen a couple mentions of using Datomic's tx-report-queue to reactively rerun queries, however it seemed to me that it was more of a fun trick that wasn't expected to scale, since every active query would have to rerun on every transaction. Is this the case, or are you using something like this in anger? (This is the first I've heard of your team using Datomic onprem specifically.)
I would've assumed this would require filtering new transactions for only those that would apply to each individual query, although perhaps Datomic's cache is efficient enough to do this for a reasonable amount of active queries?#2024-02-0102:22Dustin Getznobody has come to us yet with this problem in practice, when a serious project presents with a real perf issue we will help work out the patterns. Remember, you can use missionary locally at any point to do things like locally throttle. even in a chat app like slack, most views are not realtime#2024-02-0102:23Dustin Getzand real-time apps have event based data layers anyway not databases#2024-02-0102:24Garrett Hopper👍 All fair points#2024-02-0102:25Garrett Hopper(Probably a case of trying to prematurely optimization on my part)#2024-02-0120:12zeitsteinQ re: RCF: I have test running set up as described in https://github.com/hyperfiddle/rcf?tab=readme-ov-file#ci. However, when I run clojure -X:test ... the process/terminal hangs with all tests passing. So, if I e.g. want to run tests as first of multiple scripts, the following scripts don't execute. Any ideas?#2024-02-0120:13Dustin Getzi haven't seen that before, can you send us a minimal repro?#2024-02-0120:15zeitsteinDefinitely something in my code, I isolated one src dir as problematic.#2024-02-0120:25Dustin Getzare you using async rcf? maybe you hung a test#2024-02-0120:25Dustin Getzall assertions passed but it didn't complete#2024-02-0120:30zeitsteinNo. I think I have it:
(ns foo
(:require
[xtdb.api :as xt]
[hyperfiddle.rcf :refer [tests]]))
(def node (xt/start-node {}))
(tests
1 := 1)
;; hangs without this
#_(.close node)#2024-02-0121:08zeitsteinA hacky workaround:
;; at end of ns, assuming no other test-generating ns uses `node`
(when (System/getProperty "hyperfiddle.rcf.generate-tests")
(println "closing node")
(.close node))
#2024-02-0121:11zeitsteinI don't know enough about these things, but it seems to me this is not an RCF issue.#2024-02-0121:13Dustin GetzI logged a ticket here: https://github.com/hyperfiddle/rcf/issues/83#2024-02-0121:14zeitsteinThat makes sense.#2024-02-0123:12refsetI'm well out of context, but this issue & comment is possibly relevant https://github.com/xtdb/xtdb/issues/1882#issuecomment-1387131584#2024-02-0713:32zeitsteinThis seems to work well both for convenient REPL development (as long as you run the tests manually or through rcf, i.e. clojure.test/run-tests will close the node) and through test runners (exits properly).
(ns test
(:require
[xtdb.api :as xt]
[clojure.test :refer :all]
[hyperfiddle.rcf :refer [tests]]))
(def node (xt/start-node {}))
(xt/submit-tx node [[::xt/put {:xt/id 1}]])
(xt/sync node)
(use-fixtures :once
(fn [t]
(t)
(.close node)))
(deftest bar
(is (= (xt/entity (xt/db node) 1)
{:xt/id 1})))
(tests
(xt/entity (xt/db node) 1) := {:xt/id 1})#2024-02-0200:55joshchoi have a sick idea but i am waiting till differential electric drops#2024-02-0415:58Tommy JollyboatIf it would work with a "Functional Core, Electric Shell" type architecture, you could develop the core first in Clojure, which is pretty much what I've been doing to debug more easily and swerve compile times#2024-02-0419:30xificurCGreat architecture name!#2024-02-0309:18danbuneaHi, I updated the demo for authentification with Auth0, now the ring middleware works as I intended it.
I alo added a description about how to create a free account on Auth0 and test it:#2024-02-0309:19danbuneaThe documentation is in the readme https://gitlab.com/danbunea/electric-ring-auth0-authentification#2024-02-0315:28Dustin GetzWe'll happily add this to electric-fiddle when we find time, will you license it into the public domain?#2024-02-0316:11danbuneaSure. Just tell me how to do that please, no idea 🙂 #2024-02-0316:12Dustin GetzOne way is to add this file to your repo : https://github.com/mawww/kakoune/blob/master/UNLICENSE#2024-02-0316:51danbuneadone https://gitlab.com/danbunea/electric-ring-auth0-authentification/-/blob/main/UNLICENCE?ref_type=heads#2024-02-0317:49Dustin Getzhere is the integration point for anyone following along - it took some effort to find https://gitlab.com/danbunea/electric-ring-auth0-authentification/-/blob/main/bases/server/src/server/electric_server_java8_jetty9.clj?ref_type=heads#L110#2024-02-0311:05braai engineer(Edit: resolved)
Set up a fresh Macbook and cloned electric-fiddle. Getting this error when I try to run clj -A:dev :
No version is set for command clj
Consider adding one of the following versions in your config file at
clojure 1.11.1.1208
I don't think this is an Electric Clojure issue, but has anyone encountered this?#2024-02-0311:07braai engineerInstalled Clojure as per Clojure docs via brew install clojure/tools/clojure#2024-02-0311:12braai engineerHmm, might be related to asdf.#2024-02-0311:17braai engineerResolved by running asdf global clojure 1.11.1.1208#2024-02-0417:17weicongrats on your new macbook!#2024-02-0312:21braai engineerWhat to use for routing now that history is deprecated? electric-fiddles has pages thing but it seems to be tied to fiddles. electric-starter-app doesn't seem to have any routing. Is there a guide or an example lying around using the latest Electrirc, please?#2024-02-0313:29Dustin Getznothing has changed#2024-02-0503:06Dennis Orsiniis there a good example of how i can embed electric into a react site, I've tried using useEffect to load the main script after the target div#id has rendered but electric crashes stating that dom/node is nil. This is how im binding dom/node:
[dom/node (js/document.querySelector "#searchBar")]#2024-02-0508:43Geoffrey GaillardHave a look a this https://github.com/hyperfiddle/electric-fiddle/blob/main/src/electric_tutorial/demo_reagent_interop.cljc.#2024-02-0508:45Geoffrey GaillardI understand want to do the exact opposite. But the idea is symetric.
You want a React component with a lifecyle.
• allocate a dom node (e.g. a div)
• start electric on componentDidMount
• stop electric on componentWillUnmount#2024-02-0508:48Geoffrey GaillardThis is what our https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_fulcro_dom_adapter.cljc does. You’ll have to adapt it if you don’t use fulcro.#2024-02-0523:07Dennis Orsini@U2DART3HA Thanks for the quick response, but let's say I want to put the bundled electric js code as a script tag in a Next.js app. Can I do this without wrapping the app and serving it together from a CLJS context? For example, can I add it as a script tag and do some steps onload from the vanilla js side of things?#2024-02-0610:08Geoffrey GaillardYes, it shouldn’t be an issue.#2024-02-0604:39VincentHyperfiddle's Electric still dope, a progress update and more:#2024-02-0604:49VincentElectric is still dope, the saga continues.
The engineering team at http://Shomp.co has bloomed from zero engineers to one. That is, zero-indexed array-counting, from just myself to myself and Kurt who is absolutely amazing.
Prototyping is super fast with electric, and I have been able to bring an experienced engineer up to speed in one zoom call. It's miraculous.
Of course, now that we are actually coordinating on work we are making more strategic development priorities/prioritization.
Right now we are working on integrating some cljsjs libraries and are having mixed success loading the javascript from the index dot html while also requiring the cljsjs namespace which ought in theory be redundant but there are difficulties or challenges present in invoking the constructor for the cljsjs stuff in the / via the native clojure cljc file
So from my personal experience some amazing takeaways:
+ getting a senior engineer up to speed with electric, especially if they have used clojure(script) and some sort of SPA (single page application) framework [fb/"React" or whatever] takes as many zoom calls as it takes to cover the fundamentals which is typically one.
+ Wolking through an elaborate code demo or a living project (like my actively developed application Shomp) is a great way to get someone up to speed fast and you can focus on the salient changes from what the norm was.
+ When recruiting for help, get people excited about the mission and the vision.#2024-02-0609:42danbuneaUnit testing:#2024-02-0609:43danbuneaHow can I run this (stolen) example:
(tests "switch on dom effect"
(def !div (atom nil))
(def !x (atom true))
(with (e/run
(e/client
(binding [dom/node (.-body js/document)]
(dom/div
(reset! !div dom/node)
(dom/text "a")
(dom/text (if (e/watch !x) "b" "c"))
(dom/text "d")))
(.-textContent @!div) := "abd"
(swap! !x not)
(.-textContent @!div) := "acd"
(swap! !x not)
(.-textContent @!div) := "abd"))))#2024-02-0609:46xificurCin a browser connected cljs REPL or with a cljs test runner. The electric main repo has some https://github.com/hyperfiddle/electric/blob/master/ci/run_tests_browser.sh#2024-02-0609:46xificurCbtw ctrl+shift+enter or /snippet to insert a code snippet in slack#2024-02-0609:46danbuneathanks I'll read it now#2024-02-0614:58s-olHey, trying to update from electric 428 to 540, following current electric-starter as my setup reference. I think I'm close but something seems to go wrong in the late client boot:#2024-02-0615:00s-ol1. server starts and build succeeds
2. > 👉 http://0.0.0.0:8080
3. server prints (println) up to first (e/client ...) in my electric entrypoint
4. client prints shadow-cljs: #3 ready!, Connecting...., Connected. but not (println) in first (e/client)#2024-02-0615:03s-oli.e. with my entrypoint (`seshat.core/App`):
(e/defn App [ring-request]
(println "who?")
(e/server
(println "server"
(e/client
(println "client")
....
and src-dev/dev.cljc:
#?(:cljs ;; Client Entrypoint
(do
(println "client-entry")
(def electric-entrypoint (e/boot-client {} seshat.core/App nil))
(defonce reactor nil)
(defn ^:dev/after-load ^:export start! []
(println "starting")
(set! reactor (electric-entrypoint
#(js/console.log "Reactor success:" %)
#(js/console.error "Reactor failure:" %))))
(defn ^:dev/before-load stop! []
(when reactor (reactor)) ; stop the reactor
(set! reactor nil))))
on the server I read
> who?
> server
and on the client
> client-entry
> browser.cljs:20 shadow-cljs: call dev/start!
> dev.cljc:48 starting
> dev.cljc:49 Connecting...
> electric_client.cljs:117 Connected.#2024-02-0615:04s-olattempt checked into branch electric-540 of my repo, @U09K620SG and @U09FL65DK you still have pull access (git@git. MY-NAME .nu:seshat.git)#2024-02-0615:06Dustin Getzpaste your app root e/fn into a clean starter app and see if it works#2024-02-0615:07s-olyou mean with all my other ns etc that it references?#2024-02-0615:08Dustin Getzthe code snippet pasted above does not reference any other namespaces#2024-02-0615:08s-olit does below, but we never get therer#2024-02-0615:08s-olok, so just check printing, sure#2024-02-0615:16s-olokay, some progress - i removed all my old code from App and have only exactly this:
(e/defn App [ring-request]
(println "server")
(e/client
(println "client")))
and that works as expected#2024-02-0615:18s-olhowever adding dom:
(e/defn App [ring-request]
(println "server")
(e/client
(println "client1")
(binding [dom/node js/document.body]
(dom/header
(dom/h1 (dom/text "seshat banking inc"))
(dom/span (dom/text ""it's only money"")))
(println "client2"))))
I see a quick flash of the dom, then I get a reactor error#2024-02-0615:18s-ol#2024-02-0615:18s-ol(the header is visible for a frame and disappears)#2024-02-0615:20Dustin Getzwhat is this ""it's only money""#2024-02-0615:20s-olserver log:
#error {
:cause "Index 0 out of bounds for length 0"
:via
[{:type java.lang.ArrayIndexOutOfBoundsException
:message "Index 0 out of bounds for length 0"
:at [clojure.lang.RT aget "RT.java" 2377]}]
:trace
[[clojure.lang.RT aget "RT.java" 2377]
[hyperfiddle.electric.impl.runtime$eval_tree_inst invokeStatic "runtime.cljc" 808]
[hyperfiddle.electric.impl.runtime$eval_tree_inst invoke "runtime.cljc" 800]
[clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
[clojure.core$reduce invokeStatic "core.clj" 6885]
[clojure.core$reduce invoke "core.clj" 6868]
[hyperfiddle.electric.impl.runtime$parse_event invokeStatic "runtime.cljc" 842]
[hyperfiddle.electric.impl.runtime$parse_event invoke "runtime.cljc" 840]
[clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
[clojure.core$reduce invokeStatic "core.clj" 6885]
[clojure.core$reduce invoke "core.clj" 6868]
[clojure.core$partial$fn__5910 invoke "core.clj" 2648]
[missionary.impl.Util apply "Util.java" 29]
[missionary.impl.Sample transfer "Sample.java" 84]
[missionary.impl.Sample$Process deref "Sample.java" 34]
[missionary.impl.Reactor pull "Reactor.java" 183]
[missionary.impl.Reactor touch "Reactor.java" 222]
[missionary.impl.Reactor propagate "Reactor.java" 242]
[missionary.impl.Reactor event "Reactor.java" 398]
[missionary.impl.Reactor$1 invoke "Reactor.java" 480]
[missionary.impl.Relieve$1 invoke "Relieve.java" 86]
[missionary.impl.Sample$3 invoke "Sample.java" 146]
[missionary.impl.Observe$1 invoke "Observe.java" 74]
[clojure.core$comp$fn__5876 invoke "core.clj" 2586]
[hyperfiddle.electric_ring_adapter$electric_ws_handler$on_message__25601 invoke "electric_ring_adapter.clj" 160]
[clojure.lang.AFn applyToHelper "AFn.java" 156]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$apply invoke "core.clj" 662]
[hyperfiddle.electric_ring_adapter$ring_ws_handler$fn__25609$fn__25610 doInvoke "electric_ring_adapter.clj" 181]
[clojure.lang.RestFn invoke "RestFn.java" 423]
[ring.websocket$eval25348$fn__25352 invoke "websocket.clj" 13]
[ring.websocket.protocols$eval25113$fn__25114$G__25098__25123 invoke "protocols.clj" 3]
[ring.adapter.jetty$websocket_listener$reify__25960 onWebSocketText "jetty.clj" 76]
[org.eclipse.jetty.websocket.core.internal.messages.StringMessageSink accept "StringMessageSink.java" 53]
[org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler acceptMessage "JettyWebSocketFrameHandler.java" 348]
[org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler onTextFrame "JettyWebSocketFrameHandler.java" 436]
[org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler onFrame "JettyWebSocketFrameHandler.java" 241]
[org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor lambda$onFrame$1 "WebSocketCoreSession.java" 671]
[org.eclipse.jetty.server.handler.ContextHandler handle "ContextHandler.java" 1466]
[org.eclipse.jetty.server.handler.ContextHandler handle "ContextHandler.java" 1485]
[org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker$1 handle "AbstractHandshaker.java" 212]
[org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor onFrame "WebSocketCoreSession.java" 671]
[org.eclipse.jetty.websocket.core.AbstractExtension nextIncomingFrame "AbstractExtension.java" 145]
[org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension nextIncomingFrame "PerMessageDeflateExtension.java" 236]
[org.eclipse.jetty.websocket.core.internal.DemandingFlusher emitFrame "DemandingFlusher.java" 145]
[org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher inflate "PerMessageDeflateExtension.java" 487]
[org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher handle "PerMessageDeflateExtension.java" 413]
[org.eclipse.jetty.websocket.core.internal.DemandingFlusher process "DemandingFlusher.java" 169]
[org.eclipse.jetty.util.IteratingCallback processing "IteratingCallback.java" 243]
[org.eclipse.jetty.util.IteratingCallback succeeded "IteratingCallback.java" 369]
[org.eclipse.jetty.websocket.core.internal.DemandingFlusher onFrame "DemandingFlusher.java" 104]
[org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension onFrame "PerMessageDeflateExtension.java" 93]
[org.eclipse.jetty.websocket.core.internal.ExtensionStack onFrame "ExtensionStack.java" 120]
[org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession onFrame "WebSocketCoreSession.java" 481]
[org.eclipse.jetty.websocket.core.internal.WebSocketConnection onFrame "WebSocketConnection.java" 271]
[org.eclipse.jetty.websocket.core.internal.WebSocketConnection fillAndParse "WebSocketConnection.java" 464]
[org.eclipse.jetty.websocket.core.internal.WebSocketConnection onFillable "WebSocketConnection.java" 349]
[org.eclipse.jetty.io.AbstractConnection$ReadCallback succeeded "AbstractConnection.java" 314]
[org.eclipse.jetty.io.FillInterest fillable "FillInterest.java" 100]
[org.eclipse.jetty.io.SelectableChannelEndPoint$1 run "SelectableChannelEndPoint.java" 53]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy runTask "AdaptiveExecutionStrategy.java" 421]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy consumeTask "AdaptiveExecutionStrategy.java" 390]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy tryProduce "AdaptiveExecutionStrategy.java" 277]
[org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy run "AdaptiveExecutionStrategy.java" 199]
[org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread run "ReservedThreadExecutor.java" 411]
[org.eclipse.jetty.util.thread.QueuedThreadPool runJob "QueuedThreadPool.java" 969]
[org.eclipse.jetty.util.thread.QueuedThreadPool$Runner doRunJob "QueuedThreadPool.java" 1194]
[org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run "QueuedThreadPool.java" 1149]
[java.lang.Thread run "Thread.java" 840]]}
ERROR hyperfiddle.electric-ring-adapter: Websocket handler failure. nil
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at clojure.lang.RT.aget(RT.java:2377)
at hyperfiddle.electric.impl.runtime$eval_tree_inst.invokeStatic(runtime.cljc:808)
at hyperfiddle.electric.impl.runtime$eval_tree_inst.invoke(runtime.cljc:800)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
at clojure.core$reduce.invokeStatic(core.clj:6885)
at clojure.core$reduce.invoke(core.clj:6868)
at hyperfiddle.electric.impl.runtime$parse_event.invokeStatic(runtime.cljc:842)
at hyperfiddle.electric.impl.runtime$parse_event.invoke(runtime.cljc:840)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
at clojure.core$reduce.invokeStatic(core.clj:6885)
at clojure.core$reduce.invoke(core.clj:6868)
at clojure.core$partial$fn__5910.invoke(core.clj:2648)
at missionary.impl.Util.apply(Util.java:29)
at missionary.impl.Sample.transfer(Sample.java:84)
at missionary.impl.Sample$Process.deref(Sample.java:34)
at missionary.impl.Reactor.pull(Reactor.java:183)
at missionary.impl.Reactor.touch(Reactor.java:222)
at missionary.impl.Reactor.propagate(Reactor.java:242)
at missionary.impl.Reactor.event(Reactor.java:398)
at missionary.impl.Reactor$1.invoke(Reactor.java:480)
at missionary.impl.Relieve$1.invoke(Relieve.java:86)
at missionary.impl.Sample$3.invoke(Sample.java:146)
at missionary.impl.Observe$1.invoke(Observe.java:74)
at clojure.core$comp$fn__5876.invoke(core.clj:2586)
at hyperfiddle.electric_ring_adapter$electric_ws_handler$on_message__25601.invoke(electric_ring_adapter.clj:160)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$apply.invoke(core.clj:662)
at hyperfiddle.electric_ring_adapter$ring_ws_handler$fn__25609$fn__25610.doInvoke(electric_ring_adapter.clj:181)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.websocket$eval25348$fn__25352.invoke(websocket.clj:13)
at ring.websocket.protocols$eval25113$fn__25114$G__25098__25123.invoke(protocols.clj:3)
at ring.adapter.jetty$websocket_listener$reify__25960.onWebSocketText(jetty.clj:76)
at org.eclipse.jetty.websocket.core.internal.messages.StringMessageSink.accept(StringMessageSink.java:53)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.acceptMessage(JettyWebSocketFrameHandler.java:348)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onTextFrame(JettyWebSocketFrameHandler.java:436)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onFrame(JettyWebSocketFrameHandler.java:241)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor.lambda$onFrame$1(WebSocketCoreSession.java:671)
at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1466)
at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1485)
at org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker$1.handle(AbstractHandshaker.java:212)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor.onFrame(WebSocketCoreSession.java:671)
at org.eclipse.jetty.websocket.core.AbstractExtension.nextIncomingFrame(AbstractExtension.java:145)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:236)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.emitFrame(DemandingFlusher.java:145)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher.inflate(PerMessageDeflateExtension.java:487)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher.handle(PerMessageDeflateExtension.java:413)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.process(DemandingFlusher.java:169)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:243)
at org.eclipse.jetty.util.IteratingCallback.succeeded(IteratingCallback.java:369)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.onFrame(DemandingFlusher.java:104)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension.onFrame(PerMessageDeflateExtension.java:93)
at org.eclipse.jetty.websocket.core.internal.ExtensionStack.onFrame(ExtensionStack.java:120)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession.onFrame(WebSocketCoreSession.java:481)
at org.eclipse.jetty.websocket.core.internal.WebSocketConnection.onFrame(WebSocketConnection.java:271)
at org.eclipse.jetty.websocket.core.internal.WebSocketConnection.fillAndParse(WebSocketConnection.java:464#2024-02-0615:20s-olslack formatting fail, that's unicode quotes inside#2024-02-0615:20Dustin GetzAnyway I think you can bisect this by commenting out pieces of your app#2024-02-0615:21s-ol@U09K620SG its only that block i posted now, everything else is commented out or not called (although i am still requiring other stuff)#2024-02-0615:22s-olsame thing with just
(e/defn App [ring-request]
(e/client
(binding [dom/node js/document.body]
(dom/h1 "test"))))#2024-02-0615:22Dustin Getzcould be an issue with your hot code reloading setup?#2024-02-0615:23Dustin Getzclient and server must compile simultaneously#2024-02-0615:23Dustin Getz(dom/h1 "test") is missing dom/text#2024-02-0615:27s-oltrue but same issue. also just copied exact shadow-cljs.edn from starter to be sure#2024-02-0615:28s-olonly difference is that I removed electric_starter_app from all the resources/public/... paths, but the js code is being loaded so don't think that matters#2024-02-0615:29s-olelectric-starter-app references hyperfiddle.electric via git sha, i'm using 540 from maven. Is my copied code from electric-stater-app maybe mismatched?#2024-02-0615:30Dustin Getzat this point i recommend you fork the starter app and migrate your app in one function at a time#2024-02-0615:30s-oli can't figure out how to find corresponding electric or electric-starter hash for mvn versions#2024-02-0615:30Dustin GetzI can't check your configs, there's not a lot we can do for you other than wait for you to produce a minimized repro if you think you have a bug#2024-02-0615:30Dustin Getzsorry that the errors suck, IC made it worse, still working out the kinks#2024-02-0615:31s-oli copied electric_starter_app.main into my setup and it works#2024-02-0615:32s-olso must be some side effect from requiring some of my commented/uncalled code#2024-02-0615:32s-oli'll start removing imports until stuff starts working#2024-02-0615:32Dustin Getzyes#2024-02-0615:33Dustin Getzit was super not clear that you had a bunch of requires as part of this reproduction#2024-02-0615:33xificurCindex out of bounds errors typically mean misaligned server/client code. Known reasons for that
• different versions compiled on server/client. This can be a case from e.g. browser cache or evaling a change in the CLJ REPL but not saving it (shadow not recompiling)
• peer-dependent macroexpansion, e.g. assert elided on client and kept on server
• some users reported tools.namespace refresh issues
• conditionally requiring electric code, e.g. (:require #?(:cljs my-electric-ns))#2024-02-0615:34Dustin GetzCompilation is e/def by e/def, so all :requires with electric code inside are in play#2024-02-0615:34s-olsorry, i wrote this earlier, but the thread got very crowded
> its only that block i posted now, everything else is commented out or not called (although i am still requiring other stuff)
#2024-02-0615:34Dustin Getzthings like #?(:cljs (e/def ...)) are not allowed#2024-02-0615:36Dustin Getz*if they contain e/server code#2024-02-0615:36s-oli have one of those but it doesn't have server code#2024-02-0615:36Dustin Getz(:require #?(:cljs your-ns-with-electric-code)) is not allowed#2024-02-0615:39s-olah! it's something like that (at least one of my issues)#2024-02-0615:40s-olI had: #?(:clj (:require [my-ns-with-electric-code] :refer [bindings]])#2024-02-0615:41s-olbindings is a macro, used in e/server#2024-02-0615:42s-olstill quite confused about how to deal w macros and cljc#2024-02-0615:44s-olsorry, wrote things backwards above before. CLJ macro used in e/server#2024-02-0708:07henrikHas anyone seen CLJS advanced compilation failing on seemingly random vars missing like this:
shadow-cljs - HTTP server available at
shadow-cljs - server version: 2.27.2 running at
shadow-cljs - nREPL server started on port 5555
[:client] Compiling ...
------ ERROR -------------------------------------------------------------------
File: /tmp/build/src/app/electric/transact.cljc:103:10
Error in phase :compile-syntax-check
RuntimeException: No such var: nippy/thaw
The file line and column numbers are nonsense as well. There’s no such var at the place indicated (the file is 19 lines in total). To be clear, Nippy is a dep of ours (so it’s not hallucinating, at least).#2024-02-0708:56JThis error occur with normal compilation?#2024-02-0709:02henrikNot in dev no, this is strictly with advanced compilation, local as well as CI. It’ll randomly work sometimes.
I saw the same problem when initially switching to Electric with incremental compilation. I went around and (it felt like) randomly moved things around in the codebase, and the problem disappeared.
The background is that I’m working on a commit where I’ve thrown out our old (non-Electric) code, and this problem has now reoccurred in this branch. It includes no changes to the Electric part of the app, but is a deletion of ~200 files and ~30k lines of code of non-Electric stuff.
The fact that I saw it the first time when switching to IC hints that it might be Electric related, but the fact that I haven’t touched Electric code in this branch indicates that it might be something else.
I’m a bit lost as to how to pin down what the source of the problem is.#2024-02-0709:04JThis is very strange indeed. The CI has cache or something like that?#2024-02-0709:05henrikYeah, normal Docker cache stuff for deps, but not for code. Also, since I’m seeing the same thing locally, where I can control caching entirely, indicates that something else is at foot.#2024-02-0709:06JDo you use Rama? I have similar issue with IC and Rama together.#2024-02-0709:08henrikAha, interesting to hear. No, not in this branch.#2024-02-0709:10henrikThe randomness could indicate that it has something to do with load ordering. It happens in the CLJ phase of Shadow-cljs compilation anyway.#2024-02-0709:20henrikSome more examples. What fails exactly differs from time to time, but it all kind of seems to bottom out in libraries rather than our own code.#2024-02-0710:29henrikOn closer inspection, all errors seem to be complaining about CLJC files of this nature:
(ns some-ns
#?(:cljs (:require-macros some-ns))
#2024-02-0711:11Dustin Getzplease confirm - can you ever repro locally (not CI) with deleted shadow caches?#2024-02-0711:19henrikYes, it’s reproducible locally with a brutally clean repo.#2024-02-0711:19henrikBut only advanced compile, it doesn’t occur in dev.#2024-02-0711:20henrikI’ve managed to proceed by removing :require-macros and doing some-ns/macro-fn instead. This eliminates that particular NS from the pile of possible error candidates.#2024-02-0711:21henrikI suspect that if I refactor the rest, it will go through predictably. I’ll confirm and get back to you.#2024-02-0711:21Dustin Getzplease post shadow version #2024-02-0711:21henrik2.27.2
#2024-02-0711:23henrikRandom fact: our app weighs in at ~13mb uncompressed (2.9mb compressed)#2024-02-0711:24henrikAn early theory was that it was some OOM and failing to load nses during compilation, but it seems that is not it.#2024-02-0711:24Dustin Getzi don’t recall seeing any issues that match this previously but IC has not been out very long, i’ll get back to you#2024-02-0711:25henrikI encountered the same when initially switching to IC.#2024-02-0712:03xificurCI've seen similar error beforehttps://github.com/hyperfiddle/electric/blob/ae1341bc8c5566b45179c9407bc9d902e91c0847/src/hyperfiddle/electric/impl/expand.clj#L32#2024-02-0712:04siddharth yadavI also encountered compilation errors when updating to IC with this sort of cljc code
(ns some-ns
#?(:cljs (:require-macros some-ns))
but in my case it turned out my name space was wrong.#2024-02-0712:22henrikI’ve now removed these in favour of direct refers, and it looks like I can now compile it consistently. The seeming “soft limit” on this is confusing, it seems like the kind of thing that should fail consistently or not at all. Not trigger after some level of saturation.#2024-02-0712:32Dustin GetzPlease can you copy paste an example of exactly what you had at first and what you changed it to?#2024-02-0713:14henrikSure. So for example (where text-area-primary is a macro):
[app.electric.component.text-area :refer [text-area-primary]]
…
(text-area-primary …)
To
[app.electric.component.text-area :as text-area]
…
(text-area/text-area-primary …)
And in text-area ns:
From:
(ns app.electric.component.text-area
#?(:cljs (:require-macros app.electric.component.text-area))
(:require …))
To
(ns app.electric.component.text-area
(:require …))
#2024-02-0713:34henrik~Bundle size seems to have shrunk as well. Hang on, I’ll confirm.~ False alarm, I was comparing a compile with debug symbols to one without.#2024-02-0715:04Dustin GetzSo in principle we should be able to reproduce this in starter app, the problem is nondeterminism – only shows up in large electric codebases?#2024-02-0715:04Dustin GetzPlease say more about the "soft limit" idea, I didn't follow that#2024-02-0715:14henrikI’m not saying there’s an actual “soft limit”, it’s more of an expression of my confusion. This pattern of requiring macros was repeated pretty often in the codebase, ~15 times. So:
1. Why did I not see this problem since the initial IC migration, and only now after removing unrelated code?
2. Removing one instance of this pattern of macro requirement excluded that instance from showing up (but there were still problems).
3. Removing a second one dramatically increased the chances of getting a compiling build.
4. Removing all of them now seems to produce predictably successful builds.
Did I happen to refactor the two instances that actually had some kind of problem with them? Or could it have been any of them, just that I’m decreasing some kind of pressure somewhere by omitting them?#2024-02-0715:19henrikThe app size is pretty substantial, so it’s not impossible that there’s some path of graph expansion would lead to a giant tree which subsequently blew some fuse, making the compiler blame the nearest thing at hand
(The term “graph expansion” used loosely; I don’t understand the particulars well enough to be precise in my terminology).#2024-02-0715:20henrikBecause come on shadow; blaming internals of XTDB to be missing while compiling CLJS? Really?#2024-02-0715:21Dustin GetzOk, and just to confirm, the reproduction always involves :require-macros in the module ns and :refer [my-macro] in the consuming ns?#2024-02-0715:21henrikYes, that seems to be the culprit. Backing to a commit where this hasn’t been refactored promptly reintroduced the problems.#2024-02-0715:22henrikAnd the intervening commits contained only refactoring of this.#2024-02-0715:24henrikBtw @UHZPYLPU1, depending on how you require your Rama stuff (which is a giant pile of macros), refactoring this might eliminate your problems.#2024-02-0715:27henrikNot to mention it would be interesting to have the pattern confirmed if it is indeed the same problem that you saw.#2024-02-1219:41Dustin GetzSo did you get the 30k LOC deleted?#2024-02-1307:30henrik#2024-02-1308:42henrikIt landed at 20k in the end, but still a relief.#2024-02-0715:06KetanI've read through the missionary docs and I think I have a pretty good handle on it now.
Is there any documentation that goes into detail on how electric and missionary interact, maybe some interop example?
One specific question I had is: (e/fn [] ...) produces an electric flow, but is this a stream, signal, or can be either depending on the body?#2024-02-0716:50Dustin Getz(e/fn [] x) will type check as a missionary continuous flow, which is to say that it is a flow which has an initial value and is defined for all time (always has a most recent value). Strictly speaking, "continuous flow" and "signal" are not quite the same thing. In missionary the word "signal" means: a continuous flow that has been memoized. m/signal is the missionary operator which allocates and supervises the memo buffer. Strictly speaking, (e/fn [] x) is not a signal, it is just a continuous flow. Electric let is the electric operator which compiles down to m/signal (i.e., all electric let bindings are memoized).#2024-02-0716:51Dustin GetzMy statements to beginners regarding missionary signals may be blurring the distinction between signal and continuous flow, which is hard to avoid when matching people's priors from other libraries which may not separate or cleanly distinguish between the two#2024-02-0716:54Dustin GetzI don't think we have clear docs currently about the interaction; we used to have some examples but they are "in transition" and need to be checked and republished#2024-02-0716:54Dustin GetzThe electric lang test suite is a great place to look#2024-02-0716:56Dustin Getzbig picture: as you know, e/fn returns a value that typechecks as missionary continuous flow. Similarly, new—in addition to "calling"/booting an electric lambda—will accept a missionary continuous flow and join it into the electric program#2024-02-0716:56Dustin Getzi.e., you might think of this aspect of new as await on missionary flows#2024-02-0718:05KetanThank you, that helps a lot#2024-02-0720:26zeitsteinRCF Q. Is there a way to run tests in all files from the REPL? Both clj and cljs. The answers I found seem to imply "no", so just wanted to double-check.#2024-02-0721:15Dustin GetzI don't think we've ever tried to do that#2024-02-0722:33Geoffrey GaillardRCF has a clojure.test integration. You can configure it to generate clojure.test deftest and then use clojure.test/run-all-tests #"ns-regex.*"#2024-02-0722:40zeitsteinDo you mean: I could pass the JVM opt for the :dev alias and this will generate deftests?#2024-02-0722:43zeitsteinYup, that works. Thanks!#2024-02-0722:46zeitsteinAny possible footguns?#2024-02-0722:49Dustin Getzit’s meant for CI, due to mutable global namespace state i think you can dangle tests if you rename or delete#2024-02-0808:53zeitsteinTo unpack that for future reference:
• deftest has this 'dangling' problem. rcf makes it (more?) unmanageable by not having stable test names.
• Enabling rcf to emit deftest at dev time hence introduces this footgun, which doesn't exist in the usual rcf workflow.
I think this may be enough for me not to enable the integration at dev time.
I guess another option is to manually trigger a re-evaluation of nses.#2024-02-0810:43Dustin Getzyou can also do it from the CLI (which gives you clj and cljs in the same run). you could shell out from the repl?#2024-02-0810:46zeitsteinYep, yep. The motivation is actually not having to run through CLI when the cljs setup is already running (during dev).#2024-02-0720:48zeitsteinLeo is just a straight-up sorcerer, isn't he? https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/incseq.cljc#2024-02-0800:11JAtkinsEvidently haha https://github.com/hyperfiddle/electric/blob/ae1341bc8c5566b45179c9407bc9d902e91c0847/src/hyperfiddle/incseq.cljc#L128#2024-02-0810:22henrikWhat’s the semantics for a reactive loop? It seems like each call to recur generates a scope that remains active for as long as the loop itself remains mounted. Is this true?#2024-02-0810:40Dustin Getzyeah, see with-cycle if you don’t want a new frame per iteration. the debate is, consider a loop with a dom/div in the body. do you want a sequence of divs or do you want to reuse one div? there may be other aspects i’m not recalling atm#2024-02-0810:41Dustin Getzwe have more design work to do here#2024-02-0810:41Dustin Getzwhat are you trying to do?#2024-02-0811:04henrikGood question, both might be valid for different use cases. In this case I’m looping through AI generation for different sections of a page, where each iteration waits for the AI to finish generating before triggering recursion. So in this particular case, I don’t need each recursion to be distinct.#2024-02-0903:18Vincentis there an e/for ?#2024-02-0907:12henrikSame as calling for-by with identity#2024-02-0915:19Vincentcould you show me a verbose equivalence please#2024-02-0915:24henrikSure: (for-by identity [x y] …). You can also read the source:
(defmacro for
{:style/indent 1}
[bindings & body]
`(hyperfiddle.electric/for-by identity ~bindings ~@body))
As you can see it’s just calling for-by.#2024-02-1103:16Vincenteach item in coll ought be reference-able and therefore for-by makes more sense to me also#2024-02-1103:17Vincent(extracting props from the identity case was nonobvious and is still opaque 😅 )#2024-02-1208:30henrikThe identitycase does have use-cases, such as when iterating through a sequence of IDs, or other things that are expected to be unique and stable (a range, for example). Still, it’s better to be explicit about it.#2024-02-1210:47VincentI'm adding :as to my destructurings so I can pass the whole item to the next layer#2024-02-0920:54s-olIs this (e/def var (init-expr...)) still supposed to work this way?
https://github.com/hyperfiddle/electric/blob/7ef1612c9659d1e88f2f41223f5c7c045a5eb399/src/contrib/electric_goog_history.cljc#L62-L63#2024-02-0920:56s-olI have a history implementation based on electric goog history but navigation isn't working properly anymore. From breakpoints in closure library JS it seems as though new-goog-history might be getting called too often (maybe every time !history is referenced in a new context?)#2024-02-0921:07s-olseems that moving the value part of those two definitions out into nested (binding [...]) expressions (since they refer to each other) makes it work as it used to pre-IC#2024-02-1218:41Dustin Getzcould be a regression IDK i logged a ticket to take a closer look in the next couple weeks when we re-release the next set of demos in electric-fiddle#2024-02-1218:42Dustin Getzat a very quick glance it looks like a known regression in resolving e/def that I don't think we addressed yet#2024-02-1123:39simonkatzI would like to display a table that’s ~50 columns wide and several thousand rows high.
The code below is a simplified version of what I’d like to do. It displays a times table, with the max x and y values as input. The reverses are just so I can easily see when things change.
The code I have is slow to render when I get to a few hundred rows, and things break (“Aw, Snap!” in Chrome) when I try a few thousand rows.
Is there anything I can do to make this work better? (Maybe I’m doing something wrong. Maybe it’s not wrong as such but more sophistication is needed.)#2024-02-1123:41simonkatzIt’s lines 30-36 above that are the pertinent ones. The rest is just dealing with the two input fields.#2024-02-1201:17weii also had performance issue with a long table, and was recommended to look into virtual scroll https://clojurians.slack.com/archives/C7Q9GSHFV/p1706795492692999?thread_ts=1706623472.727089&cid=C7Q9GSHFV#2024-02-1201:28Dustin Getzwe virtualized our grids from day 1 and never really looked back, we have a demo somewhere of a folder tree view over node_modules, 10k-20k rows perfect performance Philadelphia to Paris#2024-02-1201:36Dustin Getzelectric is not a fast language in the way that rust is fast, it is fast inclusive of IO and compared to hand coded IO and client databases and such, but as a language it pays a heavy runtime cost (today at least). you definitely do not want to be processing large collections in electric. if you are ever forced to do that you should drop down to clojure#2024-02-1210:26simonkatzThanks; I’ll take a look at the Datomic browser.#2024-02-1210:36simonkatzHow would I “drop down to clojure” in my example?
I’ve tried this:
(defn table [max-x max-y]
(dom/table
(dom/props {:style {:white-space "nowrap"}})
(dom/tbody
(for [i (reverse (map inc (range max-y)))]
(dom/tr
(for [j (reverse (map inc (range max-x)))]
(dom/td (dom/text (* i j)))))))))
When I refresh my browser the table appears briefly, but then disappears and there’s an "Index 0 out of bounds for length 0" error.#2024-02-1212:03Dustin Getzit’s not legal to put electric code in clojure.core/for, which presumably wraps the body continuation in a clojure lambda#2024-02-1212:03Dustin Getzi’m not sure why it caused a desync, i’ll log a ticket to take a look#2024-02-1212:36simonkatzFWIW, I’m not getting the index out of bounds error anymore (in a fresh REPL). I’m getting a more reasonable No such var: hyperfiddle.electric-dom2/new-node. So maybe previously things had gotten into a weird state.#2024-02-1212:37simonkatzSo, for my example do I need to be looking at the virtualization thing? And was the “drop down to clojure” for processing large collections comment something more general?#2024-02-1213:12Dustin Getzgenerally, if you want to process 1000s of items in a loop (and do this every time something changes) you should leave electric and do it in clojure/script using defn. If you truly need to have 1000s of dom elements on the screen, electric v2 is not useful for that, you can use document.createElement etc from clojurescript, or try using a react wrapper if they are able to do it (vdom approaches have the same linear scan scalability issues but perhaps they are optimized enough that they can handle 1000 idk). Electric v3 however actually will be able to manage 1000s of dom elements efficiently without linear scans due to the differential dataflow internals#2024-02-1213:14Dustin GetzThe real answer is virtual scroll, I don't understand why anyone would ever want to have 1000s of dom elements on the page and especially in one monster collection, you start exceeding browser performance thresholds#2024-02-1213:22simonkatzOK; thank you. That’s all very helpful.#2024-02-1208:12weidoes anyone have a ui4-style contenteditable input i could borrow? i'd like a div whose text you can edit, and "input" triggers an e/fn with optimistic rendering#2024-02-1208:35weihere's my first take, but it's unusable because the cursor gets reset to the start on every edit
#?(:cljs (defn inner-html [^js e] (.-target.innerHTML e)))
(defmacro content-editable [v V! & body]
`(div (props {:content-editable true
:contentEditable true})
(control "input" inner-html identity ~v ~V! dom/set-val ~@body)
(text ~v)))#2024-02-1213:15henrikThat’s normal with contentEditable. You need to reset the cursor position on every re-render.#2024-02-1213:16henrik@ggaillard discovered something wonky in 689f5c44607d33897ccb213f887a9566e65f6f45:
base.js:650 Uncaught Error: Importing test-only code into non-debug environment: goog.testing.stacktrace
at base.js:650:15
at append.js:2:1
(anonymous) @ base.js:650
(anonymous) @ append.js:2
Points here:
goog.setTestOnly = function(opt_message) {
if (goog.DISALLOW_TEST_ONLY_CODE) {
opt_message = opt_message || '';
throw new Error(
'Importing test-only code into non-debug environment' +
(opt_message ? ': ' + opt_message : '.'));
}
};
This happens on advanced compilation only, as you might have guessed from the error. The commit just prior to this one works fine.#2024-02-1213:21henrikUnfortunately, the error is not more helpful than that. It just spits the entry point of the app (in append.js) and the Closure source location.#2024-02-1213:21henrikI tried this on shadow 2.27.2 and 2.27.3.#2024-02-1213:22Geoffrey GaillardThank you for the report. I think I know where to look at. Will keep you posted 👍#2024-02-1213:23henrikNot urgent or anything, I just discovered it when tracking master/configuring Jetty for the WSOD stuff.#2024-02-1213:29henrikBtw, the Jetty 12 release from ring-jetty9-adapter has been updated with :ws-max-binary-message-size and :ws-max-frame-size, which means it now exposes the knobs needed to run Electric according to spec.#2024-02-1213:30henrikhttps://github.com/sunng87/ring-jetty9-adapter/releases/tag/0.32.0#2024-02-1216:23Geoffrey GaillardFixed on 0cfb98429f0e9b4cb7822173d55d1498ce2dd972#2024-02-1307:28henrikConfirmed; it works fine.#2024-02-1219:33Alan CarrollEight months later I finally finished putting together a demo repo for combining electric with https://github.com/cjohansen/portfolio if anyone has interest https://github.com/alan-carroll/electric-portfolio#2024-02-1407:08danielstocktonIs differential electric already out there in some form to play with?#2024-02-1411:31Dustin Getznot yet#2024-02-1415:55Tommy JollyboatIn (e/for-by f [x xs] (dom/li ,,,)), I'm unsure of what f does, and hence why e/for-by identity leads to excessive recomputation. My guess is that for-by gives each child (dom/li in this case) an invisible ID that is equal to f(x), so the dom node (or whatever) can be re-used when x changes, so f = identity would entirely replace our dom/li when any aspect of x changes, instead of re-using the existing node (/child). Am I close? Why doesn't e/for-by (constantly nil) break?#2024-02-1416:03Dustin Getzyes, if you have used reagent or reactjs this is the “key” #2024-02-1416:04Dustin Getzhttps://stackoverflow.com/questions/28329382/understanding-unique-keys-for-array-children-in-react-js/43892905#43892905#2024-02-1416:04Dustin Getzas described here : https://electric.hyperfiddle.net/(electric-tutorial.demo-system-properties!%53ystem%50roperties)/#2024-02-1416:41Tommy JollyboatYeah i'm afraid i've not used React. I did re-read the System Properties readme, but didn't quite follow enough.#2024-02-1416:49Tommy JollyboatOne confusing thing in that readme is calling it a "reactive map operator" implied to me that it e/for-by f [x xs] binds x like e/for-by identity [x (map f xs)] , which I now know isn't true, but threw me.#2024-02-1416:49Tommy JollyboatEither way I get it now 🙂 Looks like a good default is to iterate over a vector of maps, and use e/for-by :db/id to do so.#2024-02-1416:50Tommy JollyboatThanks!#2024-02-1416:53Tommy JollyboatOh! Final Q. In Electric, presumably this "key" is on the implicit do that forms the for-by's body?#2024-02-1417:37Dustin GetzYeah, the for-by body is wrapped into an e/fn, and the lifetime of that e/fn is controlled based on that key#2024-02-1508:01wei(edit) TLDR: the issue was that save changes (state/get-doc) which creates a new atom. wrapping with e/snapshot fixed the issue. updated the gist
not sure if this is an electric question or a prosemirror question. here's my setup for an rich text editor that saves on change:
(e/defn Saver [!doc]
(e/client
(let [doc (e/watch !doc)]
(e/discard
(e/server
(e/discard
(state/save-doc! (e/client (serialize-node doc)))))))))
(e/defn Editor [& [width]]
(e/client
(let [schema (create-schema)
!doc (atom (deserialize-node schema (e/server (state/get-doc))))]
(Saver. !doc)
(div (div (props {:id "proposal-editor" :class "prose"}))
(div (props {:id "content"}))
(println "Rendering editor view")
(EditorView.
(util/get-element-by-id "proposal-editor")
#?(:cljs #js {:state (EditorState.create. #js {:doc @!doc :plugins plugins})}))
nil))))
I have a plugin that updates the !doc atom which triggers Saver to persist the doc to the server. it seems to work except that every change produces a new instance of the editor (see video). I can make the issue go away by commenting out this line:
(Saver. !doc)
what is the Saver electric function doing that might be messing with prosemirror? "Rendering editor view" only gets printed once, so I don't think re-rendering is causing the issue.
full code in case it's helpful: https://gist.github.com/yayitswei/eb1796428911db7865ff0c310ce80614#2024-02-1508:10henrikMy guess would be that (state/get-doc) updates reactively? Try wrapping it with e/snapshot.#2024-02-1508:13weiwow that did it.. amazing, thank you!#2024-02-1508:14weiwhy does the println not trigger then?#2024-02-1508:15henrikIt doesn’t trigger at all? That I wouldn’t know. It looks like it should trigger exactly once.#2024-02-1508:15henrikIf you mean (println "Rendering editor view")#2024-02-1508:15weioh yeah it triggers once. i thought if the atom updated, it would trigger again#2024-02-1508:15weiyes#2024-02-1508:16henrikNo, it has no dynamic dependencies, so it won’t trigger again.#2024-02-1508:17weii see, so reactivity is fine-grained within an e/fn#2024-02-1508:17henrikYes, it’s very precise. Hence the need for the occasional snapshot to escape from it in situations like these.#2024-02-1508:17weidoes @!doc create a dependency, or does that just get evaluated once?#2024-02-1508:19henrikUnless the atom instance changes, it won’t. This was the case for you: get-doc updates, which means a new atom is created, which triggers the deref, since the this is now a new atom. Now your atom is stable, so the deref won’t re-run.#2024-02-1508:20weiaha, makes sense#2024-02-1508:22weithanks again for your help. so this is single-player which is sufficient for now. any high-level tips or resources for making it multiplayer? prosemirror seems to have an opinionated way to do it, but i haven't tried using it with electric yet#2024-02-1508:30henrikI don’t know how you can make it work for multiplayer without using a CRDT. You can make it “safe” for multiplayer by locking the text field if someone is editing it.#2024-02-1508:30henrikI.e., guarantee that only one person ever edits the text field at the time, semaphore-style.#2024-02-1508:35weimakes sense. locking would work for now i think. i'd need a timeout to release the lock, right?#2024-02-1508:36weior, i could lock it if someone just has that page open, and release on page close#2024-02-1508:39henrikOr when the editor has focus. But yeah, you’d probably need a timeout anyway, in case someone clocks out for the day and leaves the cursor in the field.#2024-02-1508:39henrikOr a way to “steal” the semaphore#2024-02-1508:43weiyup that'll work#2024-02-1508:43weii'm hoping someone publishes a electric+CRDT example I can study someday 😀 in the meantime this will def work for my purposes 🙏#2024-02-1520:35Kurt HarrigerFunny you say that. Several years ago I was working at Atlassian implementing collaborative in cljc for editing in for prosemirror specifically to support migration to confluences new editor that replaced xwiki. Unfortunately its closed source and somewhat non-trivial especially table.s However prosemirror was pretty easy to work with as the events are fairly semantic and I remember seeing a couple open source libraries collaborative editing libraries although in javascript. I’ve not looked into any of them too deeply but this one looks promising https://github.com/inkandswitch/peritext https://www.inkandswitch.com/peritext/#2024-02-1520:39Kurt Harrigeror maybe https://github.com/yjs/yjs-demos looks more active and recent#2024-02-1521:21weiThanks for the resources and validation of prosemirror. I actually switched over from lexical and the API feels much simpler.#2024-02-1607:51henrikYou probably saw that I recently wrote a collaborative text editor in Electric + Peritext recently. I’m pretty sure it’s two firsts: 1) Collaborative RT in pure Electric, 2) Peritext with blocks, but so it’s also at an early stage at the moment.
For getting something out quickly, Yjs and Prosemirror ought to be one of the safest choices, despite the edge cases and problems it has.#2024-02-1613:31weiyes, i remember commenting on it. very impressive demo!#2024-02-1603:50VincentIn our prod deployment I would like to detach the screen (e.g. ctrl+a, ctrl+d)
does prod block this ability intentionally or is ubuntu being goofy#2024-02-1603:56Kurt HarrigerPerhaps this is useful https://askubuntu.com/questions/1012530/screen-not-detaching-with-ctrla-d#2024-02-1608:30VincentOkay I think it's working now, ty#2024-02-1615:23Dustin GetzWe launched a new website https://www.hyperfiddle.net/ please reply with your feedback/reactions good and bad! Do you understand our mission? Do you understand the product?#2024-02-1616:30JGreat page, very clear! Maybe center the whole content for a better reading.#2024-02-1616:56nivekuilplease make the gif a video or something so it doesn't reset when i'm trying to read it#2024-02-1616:59nivekuilyou talk about java/js functions, what does "direct classpath linking" mean to a js programmer#2024-02-1617:00nivekuil"no GraphQL" but in the previous section you say "GraphQL-like specification DSL"#2024-02-1617:01nivekuilyou underline "electric grid" but this is out of sync with the previous section. I understand "zero api coding" to address an existing, common problem, but "electric grid" is particular and abstract. maybe make that headline more immediately meaningful#2024-02-1617:03nivekuilyou should probably expand the DAG acronym somewhere, people like the idea of graphs#2024-02-1617:03nivekuilI'm not sure "No, this is not RPC" deserves to be bolded. we all hate "web framework glue code" but most people have probably never even used rpc#2024-02-1617:05nivekuilshouldn't the footer section (""Hyper" means interconnected. "Fiddle" means play.") be near the top? also the exclamation point in "Make serious cloud datagrid GUIs like this with minimal prior coding experience!" looks amateurish#2024-02-1617:06nivekuilnot sure how much feedback you want on visuals, it looks terrible on a wide screen and there's no visual hierarchy#2024-02-1617:07Dustin Getzgreat feedbacks thank you#2024-02-1617:07nivekuilI also think the iceberg looks out of place, didn't you used to have a line about fun being important#2024-02-1617:07nivekuilyou could re-add that to foreshadow/underscore the theme#2024-02-1617:14nivekuilI feel like with how dynamic electric is, the visuals could show a lot more "juice" (output/input ratio). like instead of 4 pictures of spreadsheets, something more exciting like a carousel of videos showing code being typed and showing multiple windows synced up in real time changing the UI#2024-02-1617:14nivekuildoesn't have to be how it actually looks, just the feel you're trying to get across#2024-02-1617:33nivekuilI thought hyperfiddle's mission is to turn work into play (and thereby reducing the marginal cost of development to zero or negative) -- getting useful results quickly and with no compromise in power. I think the current landing page is less compelling than that#2024-02-1617:51Daniel Basner@U09K620SG who is your target audience for this page? What is their background?#2024-02-1618:10Dustin Getzthe intended persona is a savvy CTO or engineering leader or senior individual contributor, likely still in the Clojure community or at least Clojure-adjacent/familiar, at an operationally heavy business (i.e. post-revenue, not seed stage). The persona is slightly out of focus because the webpage is also for seed investors#2024-02-1618:25weithe pitch resonates well with me as a Clojure programmer, but I'd suggest a different landing page to reach the Retool/Google Sheets crowd (which is probably a more lucrative market atm)#2024-02-1618:25Dustin Getzwe are not ready for them yet#2024-02-1618:31Dustin Getz@U797MAJ8M i made some quick tweaks based on your feedback, thank you, and I will think more deeply about restructuring for a future larger rework#2024-02-1618:32weiif you're raising a round, can i be in it?#2024-02-1621:30Ahmed HassanIMHO, It seems like more examples would make the website better and more understandable. There should be either videos linked in each section, or it should provide link to separate page which explains things in action related to that section, to bridge gap between sales pitch and technical understanding.
HTML for datagrids is good example of showing how data structure translates to the UI, and similar video would help BYO Backend section.#2024-02-1615:33Dave MaysI feel like the ideal world would be a software combination allowing an Datalog based graph database like Asami database that runs both locally and the server, and that allows automatic sync between them as well as between other users opening up local-first applications in addition to the existing awesome multiplayer feature. It seems like maybe this would be possible with a combination of Asami and Electric and maybe the ideas behind Automerge. I saw this article from the Automerge creator Martin Kleppman talking about the possibility of CRDTs as Datalog - and made me think if those principles somehow overlapped enough with Asami one day the dream of a local-first, automatically synced, compiler managed network, graph data powerd application could be possible: https://martin.kleppmann.com/2018/02/26/dagstuhl-data-consistency.html.
(Or maybe https://github.com/cozodb/cozo would be another).#2024-02-1615:35Dustin Getz> Datalog based graph database like Asami database that runs both locally and the server, and that allows automatic sync between them
This is an open research problem with many unsolved aspects#2024-02-1615:36Dave MaysDefinitely not putting it on you guys to solve it.
If anything just saying that I believe in Electric, and that it fills a spot in my ideal world!#2024-02-1615:37Dustin GetzTo elaborate, IMO the current generation of products/startups working on this, make deep compromises which are not labeled or really visible at all in their user-facing marketing materials, probably because the teams don't even understand the implications of the tradeoffs they've made#2024-02-1615:38Dustin GetzThe local first startups in particular are, afaict, struggling to find serious commercial use cases where the local-first aspect is truly the #1 priority capability to optimize for at the cost of literally everything else#2024-02-1615:41Dustin Getzthere are some specialized market segments, like PKM, where local first seems sensible (superficially at least), but it is a tiny segment and hyper-specialized teams that work on PKM tend to build their own custom infrastructure to control the tradeoffs exactly#2024-02-1615:41Dustin Getzanyway sorry for the rant#2024-02-1615:54Dave MaysGreat rant with really good points.#2024-02-1616:14grounded_sageWhat Dustin says is true. And the only project actually attempting this is Datahike. #2024-02-1622:09wikipunkVideo games seem like a great use case / market for local first graph dbs you can write rules against with frontend sync that also makes multiplayer like baldurs gate 3 possible #2024-02-1622:42Dustin Getzmaybe? console games have persistent online connection these days, competitive MOBAs run the game on the server for security and all user interactions are streamed even mouse clicks to evaluate game actions remotely #2024-02-1622:45Dustin Getzfor a real time shooter you need optimistic updates but still the game runs on the server, outcomes can’t be decided locally anyway#2024-02-1622:46Dustin Getzbg3 has trash netcode btw it desynced or glitched a dozen times in my party playthrough, still GOTY- which belabors the point that it’s just not that important to the business#2024-02-1721:24VincentSo with XTDB new database entries would reflect in the ui immediately, but with datomic it seems that a fresh query is not triggered when new entries enter the db, and i'm wondering if that's because (d/db conn) references a database at a particular moment in time ... and therefore we should probably be doing (constantly ...) re-grip'ing the database as an electric definition?#2024-02-1721:56Dustin Getzyou need the latest-db helper in both xtdb and datomic#2024-02-1721:56Dustin Getzthere is a datomic version around if you search#2024-02-1721:57Dustin Getzsearch the slack#2024-02-1818:12VincentGotcha. That makes sense. Will seek.#2024-02-1818:20VincentOur plan is to use datomic cloud and share state across 2 electric apps 😈#2024-02-1818:29Vincent(e/watch conn will work for datascript because it's clientside, but for serverside... needs magic, is what i understand so far.#2024-02-1803:54tobiasIn electric starter app, how would I change uberjar function in src-build/build.clj so that it makes an uberjar that I can run with just plain old java -jar app.jar ? Sorry for the dumb question. I've tried a few things and just can't seem to get it right.#2024-02-1804:44Kurt HarrigerI think this should work:
Add (:gen-class) to the prod.clj ns
then modify build.clj to compile before uber
(def basis (b/create-basis {:project “deps.edn” :aliases aliases}))
(b/compile-clj
{:basis basis
:ns-compile ’[prod]
:class-dir class-dir})
(b/uber {:class-dir class-dir
:uber-file jar-name
:main ’prod
:basis basis })#2024-02-1804:45Kurt Harrigeror #?(:clj (:gen-class)) in prod.cljc#2024-02-1807:40tobiasAwesome, that works, thank you!#2024-02-1807:16braai engineerWill my Electric app compile faster if it is split into smaller namespaces?#2024-02-1813:42xificurCHot code reload yes, because cljs compilation is per file/ns. Full prod build idk#2024-02-1818:11VincentDoes electric reload every namespace? I am noticing only reloads of the main.cljc in dev and not other namespaces (*.clj), must specify maybe?#2024-02-1819:31xificurCDev hot code reload runs a shadow-cljs watch, you can look at its docs on its logic which files to reload. You can also configure it#2024-02-1809:53braai engineerIs there some documentation for r/router? I see that it takes history as an argument, but https://github.com/hyperfiddle/electric-examples-app/blob/afe4f72d4710eb9ff20a69be99fb335d9252ec29/src/wip/demo_branched_route.cljc#L17-L37 using deprecated hyperfiddle.history, it sometimes takes an ordinal and sometimes a keyword.
When I pass keyword or an ordinal, I get a "not watchable" error because presumably it's not a valid history thing.#2024-02-1810:06Geoffrey GaillardYou can look at how it's used in https://github.com/hyperfiddle/electric-fiddle/blob/main/src-contrib%2Felectric_fiddle%2Fmain.cljc#2024-02-1915:25JHi guys! I don't understand this:
(e/def !state (atom {}))
(e/def state (e/watch !state))
(e/defn Foo [_ _]
(e/client
(swap! !state assoc :foo "foo")
(prn state)))
(prn state) prints {} :thinking_face:#2024-02-1915:41xificurCe/defs exist on both peers. Try
#?(:cljs (def !state (atom {})))
(e/def state (e/client (e/watch !state)))#2024-02-1916:11Dustin Getzit looks like when it is ambiguous we are inconsistent in the resolution rules, i will file a bug to take a look (backlog)#2024-02-1918:18Dustin GetzGeoffrey pointed out – what's actually happening here is that the swap! and the prn are racing (the implicit do is concurrent). So there's no bug here#2024-02-1919:14JHum...why this print the state:
(e/defn Foo [_ _]
(e/client
(let [!state (atom {})
state (e/watch !state)]
(swap! !state assoc :foo "foo")
(prn state)))
;; print => {}
;; print => {:foo "foo"}#2024-02-1919:15Dustin Getzlooks good to me?#2024-02-1919:21Dustin GetzIf the q is: why does it print the state twice, it's because let has an implicit do and electric do does not currently guarantee left-to-right evaluation order as the contents of the do boot#2024-02-1919:22Dustin Getzso for undefined reasons, (prn state ran before the swap!, and then after the swap, the state has changed so prn must run again#2024-02-1919:26JOk thanks dustin!#2024-02-2000:50simonkatzIs there anything in the way of API documentation? So far I’ve been going through the tutorials and searching Slack, which has been great, but I’ve reached the stage where some more structured reference material would be useful.#2024-02-2017:06Dustin GetzNo reference yet, sorry, we understand it's important#2024-02-2021:29simonkatzOK; thanks for that. Playing with Electric is a lot of fun, and for the time being I'll continue to ask dumb questions here. 🙂#2024-02-2007:09weii have some code like this for my prosemirror instance:
(e/defn Editor []
(e/client
(div
(div (props {:id "the-editor"}))
(e/discard (initialize-editor! "the-editor")))))
is there a way to ensure initialize-editor! gets called every time #the-editor div is redrawn? i assumed the current implementation would ensure this, but in some cases i get an uninitialized blank div.#2024-02-2007:32weii made a minimal repro to illustrate my issue:
(ns mock-editor
(:require [hyperfiddle.electric-dom2 :as dom :refer [div props text hr]]
[hyperfiddle.electric-ui4 :as ui]
[hyperfiddle.electric :as e]))
;; function to mock calling some js to initialize a component
#?(:cljs
(defn initialize! [id]
(set! (.-innerHTML (.getElementById js/document id)) "Initialized")))
;; mock component that needs initialization
(e/defn Editor []
(e/client
(div (props {:id "the-editor"}))
(initialize! "the-editor")))
(e/defn Test []
(e/client
(let [!split? (atom false)
split? (e/watch !split?)]
(div
(div
;; toggle between split screen or full screen. clicking this redraws #the-editor, which then becomes an uninitialized blank div
(ui/checkbox split? (e/fn [_] (swap! !split? not)))
(text "split"))
;; press this after "split" to see the initialized version of the component (shouldn't need this)
(ui/button (e/fn [] (initialize! "the-editor")) (text "manually initialize"))
(hr)
(if split?
(div (Editor.) (div (text "another pane")))
(Editor.))))))#2024-02-2007:41wei#2024-02-2008:17xificurCcould you try to prn in the initialize! fn? Maybe it runs but set!s on the previous instance.
In this example you could pass the dom node. (div (props {:id "the-editor"}) (initialize! dom/node))#2024-02-2010:39Dustin Getzestablish an explicit dependency as Peter suggested using dom/node, electric maximizes concurrency so the div effect and the initialize effect are racing here#2024-02-2014:32weithanks! explicit dependency to prevent a race condition makes sense. Peter's suggestion fixes the issue for me. dom/node is pretty useful.#2024-02-2016:36Kurt HarrigerHmm, I’m trying to extract some common code from button handlers and find that for some reason I’m not able to pass function arguments to e/defn … made a quick branch from the starter project and having same issue. When I click the button that calls a function with an argument I get an error in the console #object[TypeError TypeError: n.call is not a function]
here is code: https://github.com/kurtharriger/electric-clojure-state-issue/compare/fn-call-error
I know electric calls e/fns with multiple args on other dom events so I’m not sure why I’m getting an error here?#2024-02-2016:39Kurt Harrigerdoh I see it now… Main etc is a defn also but I need to create an instance before calling so if I change it to (say-hello. “Alice”) it works.#2024-02-2016:41Kurt HarrigerAlthough curious why its necessary to create new instances of e/defn when calling them it seems like weird syntax, even for components Ive sometimes forgotten the ‘.’#2024-02-2016:44Kurt HarrigerPerhaps since every e/defn vars can only be used from within another e/defn the compiler could recognize these and auto new them as needed?#2024-02-2016:55Dustin GetzCompare to Reagent, which uses square brackets for reactive call semantics [F x], and React.js which uses XML for reactive call semantics <F x/> , Electric's (F. x) is the same. If it helps, you can blame dynamic typing — when passing around objects like (e/defn Foo [F] (F. x)) we don't know if F is an Electric lambda or Clojure lambda in order to automatically issue the correct call convention. There are other reasons as well that this cannot be inferred#2024-02-2016:58Dustin Getzmore discussion here: https://electric.hyperfiddle.net/(electric-tutorial.tutorial-lifecycle!%4Cifecycle)/#2024-02-2017:02Dustin GetzWe wish the error was better, in the current implementation IIUC there's just no way to detect type errors like this (thanks to Clojure). There are some tricks and heuristics but they all have gaps. We hope to revisit this after the next major version of Electric ships#2024-02-2017:32Kurt HarrigerOk thanks. In the component case it didnt seem so unnatural but didnt occur to me that I would need to new a helper function for dom action and took awhile to figure out what was wrong. #2024-02-2020:41weii'm running out of memory on the build step deploying my app to http://fly.io. using a similar setup as electric-starter-app. any troubleshooting tips? do i need to scale my build machine (currently at 4GB)?
25.77 shadow-cljs - server version: 2.26.2 running at
25.77 shadow-cljs - nREPL server started on port 7888
25.77 [:prod] Compiling ...
97.94 Exception in thread "async-dispatch-8" java.lang.OutOfMemoryError: Java heap space
#2024-02-2020:44Dustin GetzThis is surprising, you can try adding back this setting which we removed with IC because we think it is no longer needed https://github.com/hyperfiddle/electric-starter-app/commit/ec3bb1120569b2e166b156c118ffdb96aaf9ab6e#2024-02-2020:45Dustin Getzhttps://github.com/hyperfiddle/electric/issues/11#2024-02-2020:47Dustin Getzto be clear – we have not seen this on the IC branch, this issue entirely pre-dates IC#2024-02-2020:53Kurt HarrigerI was having this issue yesterday while compiling the client code (shadow-cljs build step). The issue only happened while building inside the docker container during github workflow I couldn’t replicate on my mac#2024-02-2020:54Kurt HarrigerI tried then to provision a digital ocean box and build the container there and was able to replicate on non-mac but couldn’t figure out why it was happening and tabled the issue to revisit later#2024-02-2020:55Kurt Harrigerif I tried to turn on debugging or similar I would get out of heap instead of stack overlfow#2024-02-2020:55Kurt Harrigerthe stack track was forcing a lazy sequence but I couldn’t see any non core namespaces#2024-02-2020:58markaddlemanfwiw, this is heap size problem not a stack size issue#2024-02-2021:01Dustin GetzI understand that but the error matches the issue#2024-02-2021:02Kurt HarrigerI’ll try adding the jvm-opts to my build and see if that might resolve my issue (about 10% of the time it would work and I thought I would fix it and then it would happen again)#2024-02-2021:03weii'll also try upping the heap size -Xmx3G#2024-02-2021:05weisuccessful build! thanks for the insight. now to fix the runtime issues 😅#2024-02-2021:06Dustin GetzOk, we need to understand this better. Can you paste your deps.edn as an attachment here?#2024-02-2021:07wei{:paths ["src" "resources" "src-contrib"]
:mvn/repos {"maven-releases" {:url ""}}
:deps {
com.hyperfiddle/electric {:git/url ""
:sha "f8776792bb24a3b8b81baa509e5758fea7d20d28"}
com.hyperfiddle/rcf {:mvn/version "20220926-202227"}
;; TODO should set this dependency to "provided" so it doesn't get included
;; in deploy jar
;com.rpl/rama {:mvn/version "0.10.1"}
;com.rpl/rama-helpers {:mvn/version "0.9.1"}
com.rpl/nippy-serializable-fns {:mvn/version "0.4.2"}
ring/ring-jetty-adapter {:mvn/version "1.11.0"}
org.clojure/clojurescript {:mvn/version "1.11.60"}
org.clojure/tools.logging {:mvn/version "1.2.4"}
ch.qos.logback/logback-classic {:mvn/version "1.2.11"}
datascript/datascript {:mvn/version "1.5.2"}
com.taoensso/timbre {:mvn/version "6.2.2"}
ring/ring-core {:mvn/version "1.11.0"}
ring-basic-authentication/ring-basic-authentication {:mvn/version "1.2.0"}
mount/mount {:mvn/version "0.1.17"}
com.rpl/specter {:mvn/version "1.1.4"}
org.clojure/data.xml {:mvn/version "0.2.0-alpha8"}
org.clojure/data.csv {:mvn/version "1.0.1"}
clj-commons/humanize {:git/url ""
:sha "bb51df094f7343eef331608692e0ca8d151a5c0a"}
camel-snake-kebab/camel-snake-kebab {:mvn/version "0.4.3"}
org.clojure/math.numeric-tower {:mvn/version "0.0.5"}
clj-http/clj-http {:mvn/version "3.12.3"}
net.clojars.wkok/openai-clojure {:mvn/version "0.11.0"}
com.pitch/uix.core {:mvn/version "1.0.1"}
com.pitch/uix.dom {:mvn/version "1.0.1"}
}
:aliases {:dev {:extra-paths ["src-dev" "src/test/resources"]
:extra-deps {binaryage/devtools {:mvn/version "1.0.7"}
thheller/shadow-cljs {:mvn/version "2.26.2"}
io.github.clojure/tools.build {:mvn/version "0.9.6"
:exclusions [com.google.guava/guava ; Guava version conflict between tools.build and clojurescript.
org.slf4j/slf4j-nop ; clashes with app logger
]}
org.clojure/tools.namespace {:mvn/version "1.4.5"}
nrepl/nrepl {:mvn/version "1.1.0"}
}
:main-opts ["-m" "nrepl.cmdline"
"--port" "7888"
"--interactive"]
:jvm-opts ["-XX:-OmitStackTraceInFastThrow" ;; RCF
"-Dtaoensso.elide-deprecated=true"]
:exec-fn user/main
:exec-args {}}
;:prod {:extra-paths ["src-prod"]}
:test {:extra-paths ["src-dev" "src/test/resources"]
:extra-deps {binaryage/devtools {:mvn/version "1.0.7"}
thheller/shadow-cljs {:mvn/version "2.26.2"}
nrepl/nrepl {:mvn/version "1.1.0"}
}
:jvm-opts ["-XX:-OmitStackTraceInFastThrow" ;; RCF
]
:exec-args {}}
:build {:extra-paths ["src-build"]
:ns-default build
:extra-deps {io.github.clojure/tools.build {:mvn/version "0.9.6"
:exclusions [com.google.guava/guava ; Guava version conflict between tools.build and clojurescript.
org.slf4j/slf4j-nop ; clashes with app logger
]}
io.github.seancorfield/build-clj {:git/tag "v0.8.0" :git/sha "9bd8b8a"}
thheller/shadow-cljs {:mvn/version "2.26.2"}}
:jvm-opts ["-Xmx3G"]}}}#2024-02-2021:08Dustin GetzHow many LOC in your largest namespace that contains a lot of Electric code?#2024-02-2021:08weidoes that count as an attachment? gist here https://gist.github.com/yayitswei/7996821bc8773236ce5ffa5198e25273#2024-02-2021:09wei500 LOC is the longest electric ns#2024-02-2021:09weii can break it up if it saves memory#2024-02-2021:10weibut i'm also happy paying a little extra for memory as long as it works#2024-02-2021:11Dustin Getzyou can try splitting it in half if you want, in principle ClojureScript compiles file by file, but 500 LOC is not that much unless macros are exploding it out#2024-02-2021:11Dustin Getz(500 LOC of Electric, macros that expand to Electric)#2024-02-2021:12Kurt HarrigerI updated my build alias to to include :jvm-opts [“-XX:-OmitStackTraceInFastThrow” “-Xss2m”] and the first rerun was successful. although I suspect those options are to help find the issue rather than fix it#2024-02-2021:13weito add a data point, my build failed with -Xss2m but succeeded with -Xmx3G#2024-02-2021:15weigotcha. i do have some macros but afaik nothing that explodes loc#2024-02-2021:17Kurt Harriger2 successful builds. so maybe -Xss2m did the trick for me#2024-02-2021:17Dustin GetzPlease report back if you see it again#2024-02-2021:18Dustin GetzNote that wei is using a different flag#2024-02-2021:18Dustin GetzOur records show that we've seen this error get fixed by both of these flags in different instances, I recall never understanding why sometimes one works and other times the other#2024-02-2021:20Kurt HarrigerSounds good. I may also try a small stack size and enable MaxJavaStackTraceDepth. I treid that before via JAVA_OPTS and it didn’t seem to be working didn’t occur to me to set it in the alias so maybe thats why#2024-02-2021:22weinot too familiar with uberjar, but i'm somewhat content with the explanation that building my app requires more memory than the default setting. i guess i wasn't using the full 4GB allocated by fly until adding that flag#2024-02-2021:22weiif you think the build should use less than that, happy to help troubleshoot#2024-02-2021:23weiotherwise, perhaps a comment in the readme will do to help folks hitting this issue in the future#2024-02-2021:24Kurt HarrigerI was having it fail when compiling client code via shadow-cljs which happens before the uberjar#2024-02-2021:24Dustin Getzyeah idk what is normal other than we have some larger apps that build on default settings ever since IC. We'll look harder at this stuff after Electric v3.#2024-02-2021:24Dustin GetzI did reopen the github issue, do you think that's enough wei?#2024-02-2021:25weisgtm 👍 thanks for the help#2024-02-2021:26Dustin Getzoh, what java versions#2024-02-2021:27weiopenjdk 11. here's my dockerfile, basically lifted from the starter app with an extra npm install https://gist.github.com/yayitswei/32a48c30a628e9c1224f1265451e6092#2024-02-2021:28Kurt Harrigerfirst saw issue on FROM clojure:openjdk-11-tools-deps
but I tried to upgrade to FROM clojure:temurin-20-tools-deps and still saw the issue
(had weird class not found issue using clojure:temurin-21-tools-deps so backed down a version)#2024-02-2021:29Dustin GetzOk, same as starter app#2024-02-2021:29Kurt HarrigerI also tried the latest shadow-cljs “2.27.4”#2024-02-2021:29Kurt Harrigeryeah building off fork of starter app#2024-02-2109:15avocade@U066TMAKS fwiw we’ve had to scale our Fly build machines from the start, at least 16 GB of RAM has been necessary to not get constant build crashes. For our dev/staging env builder I think at have 32 GB nowadays since many people build simultaneously on it.
Not specifically related to electric, we used uix before switching and needed at least 16 GB back then as well.
Note that our app is much bigger than the starter app, so 8 GB might be fine in your case. Also nice to use perf CPUs instead of the default shared for faster performance. #2024-02-2201:47weithat's a good data point, thanks#2024-02-2201:51weii was actually able to remove the -Xmx3G and still get it to compile. the two biggest changes i made were removing this build hook from the prod profile :build-hooks [(shadow.cljs.build-report/hook)] , and building my own docker image https://gist.github.com/yayitswei/009155f7037eba974f5aafe0d72ba336#2024-02-2201:56Kurt HarrigerMine does not have build-hooks in prod profile just :dev. I seem to have had several successful builds after adding -Xss2m but i was having inconsistent results and dont have many datapoints yet#2024-02-2201:58Kurt HarrigerIm not entirely sure how much memory github uses when running github action workflows Im just using default. Maybe also why results were inconsistent as maybe it depends on how many builds the host is running#2024-02-2220:12weiSorry to thrash on this, just reporting back that I encountered the OOM again with the above setup so I needed to add back the additional memory allocation#2024-02-2220:13wei@U06KD64RX I think you're right about inconsistency on a shared build machine#2024-02-2109:57WillHeyo! I am trying to get a table (or something that looks like it) working in electric. Found some threads discussing the topic and ended up trying to emulate what you guys are doing in the https://electric-datomic-viewer.fly.dev/(:app.datomic-browser!attribute,:abstract%52elease!name) / https://github.com/hyperfiddle/electric/blob/master/src/contrib/gridsheet.cljc. Unfortunately, even with a barebones implementation using nothing but hardcoded data, it keeps glitching out upon scrolling down (see images).
What is interesting is that if I replace the cells for-loop (https://github.com/hyperfiddle/electric/blob/master/src/contrib/gridsheet.cljc#L77) with a hardcoded div for each cell (since I know the number of columns in the test data) it will work perfectly fine. But any other way I have tried to make it work has ultimately been a failure. The value of scroll-top is jumping all over the place, so I initially thought that perhaps elements were being mounted before what was previously on the screen got unmounted (and with that bumping things out of place), but I think I was able to confirm this is not the case (with mount/unmount prints).
Maybe you have some ideas about what is happening?
I also noticed the todo deprecate comment at the top of the Gridsheet ns, and I have heard rumours about a fabled electric table you guys built. If possible, we would certainly be interested in having a look at that 👍#2024-02-2109:59WillLmk if you need any more information, or code snippets (although the barebones implementation was me going back and essentially making a carbon copy of the gridsheet code).#2024-02-2110:00WillThanks in advance 🙂#2024-02-2114:37WillDecided to reduce scope of the project in the meantime, using a paginated table view (no infinite scroll). So no rush but still keen to hear/see your take on tables in electric.#2024-02-2118:01Dustin GetzHaven't touched this in over a year but it looks misconfigured, the gridsheet requires some layout information injected from userland including a css-grid column config - https://github.com/hyperfiddle/electric-fiddle/blob/15639af98d15280d87a85af548650359660cc234/src/datomic_browser/datomic_browser.cljc#L51#2024-02-2118:04Dustin Getzwe do have feature complete and high quality application datagrids (see screenshots in our website) but they are for our consulting clients only at this point, they are not plug-n-play yet (harder to configure than the datomic browser for sure)#2024-02-2119:03WillAlright thanks, good to know. Maybe we'll be in touch in the future then#2024-02-2216:25VincentHey are any of y'all running a Datomic Ion?
We are leaning Datomic Cloud which would effectively be its own AWS endpoint that talks to a datomic instance. But there is the datomic ion where the datomic instance and your clojure app run co-locally together in that "serverless way" and I'm wondering if anyone using Electric has done this and if there are any beautiful, unexpected gotchas in the process#2024-02-2216:26Dustin GetzI don't think anybody has tried to get Electric running on an Ion, it is probably possible but expect many gotchas#2024-02-2216:27Dustin Getzthere are several threads enumerating the problem surface area#2024-02-2216:28Dustin GetzIn your case I recommend self-hosting datomic pro in the simplest way possible, it will be a much easier ops load than anything AWS#2024-02-2216:28VincentOh, this is data#2024-02-2216:28VincentI appreciate this#2024-02-2216:29Dustin Getzthere are many options to scale up Datomic Pro when you are ready, within certain scalability bounds of like 100 transactions per second sustained, at which point you will have Lots Of Money to figure it out#2024-02-2216:29VincentOkay, datomic pro running on the same digital ocean droplet as the electric app seems okay?#2024-02-2216:30Dustin Getzthat's fine for now, you'll want to run it separately at some point to not have database downtime#2024-02-2216:33Vincentgotta love that CAP theorem#2024-02-2216:34Vincentwants: snappiness and ideally no downtime#2024-02-2216:34Vincenti think can snappiness with helper db functions that i still gotta figure out for datomic, but i think i found those in the code the other night#2024-02-2216:34Vincentideally no downtime would mean a copy of the db... which sounds simple#2024-02-2216:38wikipunkI've tried to get started with electric on Ions but it was very complicated to get websockets working with authorization in the way the app expects, and haven't had time to work through the issues due to work#2024-02-2222:12VincentOkay. "A mystery" is about as good an answer as I could have hoped for at this point. Thank you @U23DRD8FN
Aiming for global availability and reliability, we are using S3 for storage, I think Datomic Pro "on prem" for the forseeable future will be plenty good as it seems to be the correct (only?) way to get good observability of the database.
So I have found these 2: https://github.com/hyperfiddle/electric/blob/0cfb98429f0e9b4cb7822173d55d1498ce2dd972/src/contrib/datomic_peer_m.clj
and
https://github.com/hyperfiddle/electric/blob/0cfb98429f0e9b4cb7822173d55d1498ce2dd972/src/contrib/datomic_contrib.clj
I think we just need to add these to our codebase, but I'm not exactly sure how these are invoked/when. Or should it happen automatically?#2024-02-2223:45Dustin Getzi don’t think you need those#2024-02-2300:59VincentOh.#2024-02-2300:59VincentDunno what is needed to get an electric def of the datomic instance then 😅#2024-02-2301:16Dustin Getzi believe you are looking for https://docs.datomic.com/pro/clojure/index.html#datomic.api/tx-report-queue#2024-02-2301:16Dustin Getzthere are a few missionary wrappers in slack somewhere#2024-02-2318:56VincentMaybe this one? https://files.slack.com/files-pri/T03RZGPFR-F05AK0ADSP9/untitled.clj#2024-02-2318:57Dustin Getzyes one of those, i cannot confirm if this is exactly what you want, you'll need to understand the discussions#2024-02-2314:46prabhaspHi all, beginner here - working with electric and datomic, and having an issue where I'm writing to the database excessively. What am I doing wrong?
Code in the thread.#2024-02-2314:48prabhaspI am working off of the latest electric-starter-app.
Db refresh is setup like this:
;; db.cljc
(defonce !db (atom nil))
(def conn)
with a modification to -main as follows
;; dev.cljc
#?(:clj ;; Server Entrypoint
(do
(def config
{:host "0.0.0.0"
:port 8080
:resources-path "public/electric_starter_app"
:manifest-path ; contains Electric compiled program's version so client and server stays in sync
"public//electric_starter_app/js/manifest.edn"})
(defn -main [& args]
(log/info "Starting Electric compiler and server...")
(alter-var-root #'conn (constantly (start-datomic!)))
(reset! !db (d/db conn))
;; datomic tx queue allows only single consumer
;; we spawn a single consumer here and feed the new db into an atom
;; clients can now watch this atom
(future (let [q (d/tx-report-queue conn)]
(loop []
(reset! !db (:db-after (.take ^java.util.concurrent.LinkedBlockingQueue q)))
(recur))))
(shadow-server/start!)
(shadow/watch :dev)
(comment (shadow-server/stop!))
(def server (jetty/start-server!
(fn [ring-request]
(e/boot-server {} electric-starter-app.main/Main ring-request))
config))
(comment (.stop server)))))
My app code:
;; main.cljc
(e/def db)
...
(e/defn View-book [book-id]
(e/server
(let [book (d/entity db book-id)
book-title (:book/title book)
; [page-id :page/text :page/number]
pages (sort-by last
(d/q '[:find ?page ?text ?number :in $ ?book-id :where [?page :page/book ?book-id] [?page :page/text ?text] [?page :page/number ?number]] db book-id))
max-page-number (apply max (conj (map last pages) 0))]
(try
(e/client
(dom/h1 (dom/text book-title)))
(e/for-by first [[page-id page-text page-number] pages]
(e/client
(dom/div
(dom/text
(str page-text " (" page-number ")")))))
(e/client
(dom/input (dom/props {:placeholder "Page text"})
(dom/on "keydown"
(e/fn [e]
(when (= "Enter" (.-key e))
(when-some [v (empty->nil (.substr (.. e -target -value) 0 400))]
(e/server
(d/transact conn [{:page/text v :page/number (inc max-page-number) :page/book book-id}])))
(set! (.-value dom/node) ""))))))
(catch Pending e (println "Loading..."))))))
(e/defn Main [ring-request]
(e/server
(binding [db (e/watch !db)]
(e/client
(binding [dom/node js/document.body]
(let [state (e/watch state!)
selected-book (:selected-book state)]
(List-books.)
(if (some? selected-book)
(View-book. selected-book))))))))
The issue is that when I hit enter on the input, that input is saved many times, as you can see in the screenshot.
Why? (I can see when we call transact, we update the watched db, which causes the d/transact to re-run. If that loop is expected, what causes this to stop?)
And what's a better pattern to use that doesn't cause an issue like this?#2024-02-2315:30Dustin Getz(without looking closely at your code) you're right that it's a cycle, e/snapshot can be used to break cycles for now. We're working on a new pure functional pattern but for now you need to understand e/snapshot. There are many threads discussing#2024-02-2315:32Dustin Getzthe d/transact side effect references max-page-number which is derived from db , there may be others#2024-02-2315:37Dustin Getz> what causes it to stop
you're racing the network latency, dom/on takes an e/fn (a "component" i.e. "piece of DAG") as a "callback"), dom/on has complicated machinery (a state machine) inside which mounts and unmounts the e/fn, managing it's lifecycle. the FSM is on the client and waiting for the first non-Pending result from the e/fn, which throws Pending at the e/server point. This is racing against the cycle between d/transact and max-page-number – and note that cycle is server-only, so you get several iterations before the client sees a non-Pending result and then unmounts the frame and tells the server to unmount as well.#2024-02-2315:39Dustin Getzwe hate the pattern, it is a "good enough in practice" until we can release a pure-functional flow-based way to deal with events – which is blocked on Electric v3 which contains upgrades needed to do this#2024-02-2317:49prabhaspThanks, that makes sense. Will try it out!#2024-02-2423:41VincentI would like a Datomic Pro starter app example.#2024-02-2501:30Vincentholy smokies i think i got it working#2024-02-2501:49VincentGot it working.#2024-02-2501:51VincentI needed 2 electric definitions that were server-biased/server-shaded
(e/def !datomic-conn (e/server (db/connection-for-shomp)))
(e/def live-db (e/server (db/next-db< !datomic-conn)))
and the (db/next-db< ...) I found via searching the slack. the method (connection-for-shomp) returns a connection but does not alter-var-root like init-conn does, as I think we can safely probably just alter-var-root the one time.#2024-02-2519:09VincentActually, wondering if there's a way to figure out
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@b5894c59
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@193a63f0
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@dcab9595
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@37730259
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@d87257cf
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@4edbb1af
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@8b4a47ca
#2024-02-2519:09Vincentunserializable reference transfer: datomic.db.Datum#2024-02-2602:23VincentMore specifically:
INFO : Unserializable reference transfer: datomic.btset.BTSet datomic.btset.BTSet@1a0d5921
INFO : Unserializable reference transfer: datomic.btset.BTSet datomic.btset.BTSet@2fb9a29
INFO : Unserializable reference transfer: datomic.btset.BTSet datomic.btset.BTSet@2abaaf15
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@12ffa88b
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@75c26b70
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@2dc871ca
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@168b806c
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@79dfc9c6
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@1fa0a466
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@7188c079
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@3b4aa6c4
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@3cdcc992
INFO : Unserializable reference transfer: com.datomic.lucene.store.RAMFile com.datomic.lucene.store.RAMFile@2645d9fb
INFO : Unserializable reference transfer: datomic.btset.BTSet datomic.btset.BTSet@5bdc1493
INFO : Unserializable reference transfer: datomic.index.Index datomic.index.Index@2d23ff47
INFO : Unserializable reference transfer: datomic.index.Index datomic.index.Index@11f95e42
INFO : Unserializable reference transfer: datomic.index.Index datomic.index.Index@25a16009
INFO : Unserializable reference transfer: datomic.index.Index datomic.index.Index@7feba63f
INFO : Unserializable reference transfer: clojure.core$eval43467$fn__43468 clojure.core$eval43467$fn__43468@76f3afef
INFO : Unserializable reference transfer: clojure.core$eval43472$fn__43473 clojure.core$eval43472$fn__43473@540b2550
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@e9847197
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@8087dcfa
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@8e382dab
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@59d71a03
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@a94d112d
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@4589dd50
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@35feabf0
INFO : Unserializable reference transfer: datomic.db.Datum datomic.db.Datum@c4eeaf14
#2024-02-2602:44prabhasp> I would like a Datomic Pro starter app example.
+1 - it took me a while to figure this out too.#2024-02-2602:45prabhasp@U055PQH9R4M - re: that serializable error - not all datomic objects can be serialized to the client. eg. datomic ids can't go down the wire.
So make sure that your client forms aren't trying to access datomic objects, like db-ids, or the return value of an @(d/transact ... etc.#2024-02-2602:46prabhaspAt least that's my understanding. I'm also a beginner at this stuff, and just started building a datomic + electric app.#2024-02-2602:49prabhaspMy understanding was picking up from these errors + https://electric.hyperfiddle.net/(electric-tutorial.demo-todos-simple!%54odo%4Cist)/
> • unserializable reference transfer - d/transact! returns an unserializable ref which cannot be moved over network, when this happens it is typically unintentional, so instead of crashing we warn and send nil instead.
#2024-02-2603:33VincentYou sly master wizard you @U06KBMJUAVC#2024-02-2603:33VincentThank you very much for that link#2024-02-2603:34VincentI am over here mulling over how nil might be the solution... starting to put on the whole amazon prime 141 movies I have on the watchlist to get to the bottom of this#2024-02-2603:34Vincentand it turns out #2024-02-2603:41VincentThanks man. That was it.#2024-02-2520:19VincentWhat is bindx 😄 ?#2024-02-2521:20VincentI was under the impression that using binding would make available a variable to all nested-scope fns below/contained within
e.g.
(e/defn BigEasyApp []
(e/server
(binding [this-is-a-server-database (get-that-database)]
(e/client
(SomeHeader.)
(SomeContent.)
(SomeFoot.)))))
but inside
SomeFoot []
gotta explicitly refer to `this-is-a-server-database`
so therefore, gotta pass `this-is-a-server-database` in args??? <
pass this in args
`this-is-a-server-database
`
results in unserializable reference transfer.
so maybe the error is with me, i need to keep the server-tint on the database reference all the way down somehow?#2024-02-2523:41euccastroTry (e/def this-is-a-server-database) at the namespace level? Then you don't need to pass it. This is how e.g. db connections are defined in the tutorials.#2024-02-2601:48Dustin Getzbind the database value in dynamic scope to an e/def, make sure to bind it on the server, and never ever touch it from the client. don’t pass it as an arg to anything except for passing it to clojure in final query, and make sure you do that on the server#2024-02-2522:21VincentThis might be an insane question, but is there a way to ensure that arguments are only passed in server-tint or client-tint ?
(CoolSiteMain. (e/server just-on-the-server) (e/client just-on-the-client)) ?#2024-02-2601:41weiwhat's a good way to get the click event for ui4/button, since the handler e/fn doesn't receive the event? i'd like to call stopPropagation on it#2024-02-2602:27VincentO_o looking at some of the other macros they expose >event with variable arity / multimethod-esque dispatch...#2024-02-2602:27Vincentif >event is not exposed i'm not really sure x.x#2024-02-2602:30Vincentadd a new macro to the ui4 file? lol xD#2024-02-2612:31Dustin Getz@U066TMAKS why do you want to call stopPropagation, IIRC the last time this was asked we solved it by disabling a containing <form> submit action#2024-02-2612:33Dustin Getzyou can also use the lower dom/on! interface for event-level operations#2024-02-2720:08weithe button is a delete button that sits on top of a clickable tab. when the delete button is pressed, i don't want the click event to bubble and activate the tab.
dom/on! works for this, thanks!#2024-02-2614:13stripyHi there, I am completely new here - just have started learning clojure a few days ago.
So is there any web games made out of Electric?
I feel fascinated about FE/BE network sync feature of Electric, I think this could be useful when making web games#2024-02-2615:43Dustin Getza couple people have tried to make games, you'll hit a problem – our lack of optimistic updates generally. (We have specific patterns for doing web forms with optimistic input controls, but it doesn't generalize to games)#2024-02-2615:45Dustin GetzWe'll be launching upgrades and improvements here after the next major version of Electric ships, which is coming in the spring#2024-02-2701:26stripyCan't wait to see upgrade, appreciate what you guys are doing#2024-02-2815:11HendrikI made a first version of electric threejs bindings. could be useful for the graphics part.#2024-02-2823:03Vincent@U023LKF2PQV could be very helpful ... https://cabbi.bo/nothing/ is a really amazing scrollable demonstration of what you can do in the basic scroll + 3D realm#2024-02-2614:38henrikI’ve packaged up the code we use for https://www.npmjs.com/package/@multiplyco/tailwind-clj into an NPM plugin, in case anyone else has encountered long Tailwind compile times.#2024-02-2800:14simonkatzIs it possible to have ui/input make changes only when the enter key is pressed, rather than on every keystroke?#2024-02-2810:08xificurCui/input no. We have ui/on-submit which is currently unused but I think it still works. You'd (dom/input (ui/on-submit (e/fn ... . You can also copy the pattern from our https://electric.hyperfiddle.net/(electric-tutorial.demo-chat!%43hat)/#2024-02-2811:05simonkatzI’ve tried a couple of things using dom/input.
Previously I had
(e/defn InputSubmit [s F]
(e/client
(dom/input (.setAttribute dom/node "value" s)
(dom/on "keydown" (e/fn [e]
(when (= "Enter" (.-key e))
(let [v (-> e .-target .-value)]
(F. v))))))))
And I’ve just tried:
(e/defn InputSubmit [s F]
(e/client
(dom/input (.setAttribute dom/node "value" s)
(ui/on-submit (e/fn [s] (F. s))))))
These differ from the tutorial examples in setting a value. (Perhaps I’m not doing that right.)
Initially, the UI fields update when their underlying value (`s` above) changes, but as soon as there has been any edit in the UI field it will stop updating.#2024-02-2811:05simonkatzHere’s a long example — a degrees Celsius to degrees Fahrenheit converter, and vice versa — where the input fields are updated when other input fields are edited…#2024-02-2811:05simonkatz#2024-02-2815:33Dustin Getzjust making sure you've seen this - https://electric.hyperfiddle.net/(electric-tutorial.tutorial-7guis-2-temperature!%54emperature%43onverter)/#2024-02-2816:35simonkatzHah! I had, but I’d forgotten about it. 🙂
I’m trying to set up something where I can make server queries only after the user presses the enter key in an input field. Is there any way to do that? At the moment I’m getting a query on every keystroke.#2024-02-2816:36simonkatzMy dom/input approach has the problem detailed above.#2024-02-2823:05VincentYou cannot capture both on-submit and dom/on key=enter, it's one or the other#2024-02-2823:05Vincentdo not differ from the tutorial#2024-02-2823:05Vincentand i think it will work XD#2024-02-2920:46simonkatz> You cannot capture both on-submit and dom/on key=enter, it’s one or the other
I am using one or the other; not both.
My example may have been over-complicated.
I’ll post a shorter, more focussed example later.#2024-02-2920:48simonkatz#2024-02-2920:50simonkatzThat’s my simpler example, above.
This code creates the following:
• An atom containing a string.
• A watch on that atom.
• A ui/button that sets the atom’s value to a random string.
• A ui/input that shows the atom’s value. The value can be edited (no need to press the enter key).
• A dom/input that shows the atom’s value. The value can be edited (the enter key must be pressed).
Note that the values displayed in the ui/input pane and dom/input pane update reactively when the atom’s value changes.
Everything works except that after the dom/input has been edited, it no longer updates reactively.#2024-02-2920:52simonkatzActually, this isn’t a big problem for me — it’s something that showed up when I was adding code to help debug things.#2024-02-2920:53simonkatzBut if someone can tell me a way to make the dom/input maintain its reactivity after being edited, I’d be interested — it might help me another time when I’m debugging.#2024-02-2923:16simonkatzI guess deleting the component and recreating it might do the job. Maybe I’ll look into that. Not sure how to do it offhand. It would be nice if there were a simpler fix.#2024-03-0211:45VincentIf you change line 43 to @!input-string does it change the behavior at all?
eg (InputSubmit. @!input-string (e/fn [s] ...))
Another idea is to change (F. v) to (F v) on line 13
[3] A final thought is to change line 13 to say (reset! !input-string v) instead of working with the (e/fn [s] ..) and relying on the (F. v) to do the trick.
A post-script (PS) final final thought is you can maiybe move lines 15, 16 (cljs def, e/def) above the InputSubmit. and see if that changes things.
I think suggestion [3] has the highest likelihood of addressing the issue#2024-03-0219:04simonkatzThanks, but none of those work and the first two make things worse. 🙂
Are you able to run the code yourself and try out your ideas? (You seem quite interested in this, and it seems to me you would get something out of running the code and fiddling with it. Up to you, of course.)#2024-03-0319:59simonkatzI’ve fixed it by changing (.setAttribute dom/node "value" s) to (clojure.browser.dom/set-value dom/node s).
Yay!#2024-03-0320:48VincentVery nice find#2024-03-0320:53Dustin Getzwhat is the difference?#2024-03-0320:57simonkatzBefore my fix, after typing into the input pane it would no longer update when the underlying atom was given a new value — the reactivity broke. (Excuse my possibly broken terminology.)#2024-03-0321:04simonkatzIs this by any chance related to the quiz question at https://electric.hyperfiddle.net/(electric-tutorial.demo-system-properties!%53ystem%50roperties)/
where an inline style gets lost?
Just wondering because I understand neither and both seem to be reactivity losses.#2024-03-0701:08Dustin Getzthe answer to the quiz is because the rendering is effectful, the new props overwrite the old props, there’s no stack #2024-03-0719:16simonkatz(Just seen this…)
It took me a while to understand what you meant by “the new props”. Looks like ui/input , via ui/control, is setting new props.#2024-02-2815:59Dustin GetzHelp me out – if you've made something with Electric, please send us a screenshot or repo link etc (DM me or reply here)! I am collecting screenshots as evidence of traction to share with investors later this year#2024-02-2816:06henrik#2024-02-2817:32Jhttps://github.com/jeans11/demo-rama-electric#2024-02-2902:48joshchoidk if this will be helpful for investors, but we made this in electric: https://youtu.be/VTDLTzLeegE?si=xKffkZB_xyBoLVVG#2024-02-2902:49joshchoedited version: https://youtu.be/dNIOPXgV53o?si=c7pTFSRRpfoYGQa4#2024-03-0116:45danbuneaMuch older version, but it shows a big benefit of Electric:#2024-03-0121:24mattiasWe're building http://uxo.ai with Electric (the app itself, not the landing page, check the video).#2024-03-0213:21mattiasI also managed to convince my consulting client (a very conservative Japanese corporation) that I can build them an internal tool for visualizing and annotating data for machine learning. Using Electric has made the codebase very compact and the app dynamic, and using XTDB has given us a way to travel back in time to see how predictions looked like for ewarlier versions of the model, which has been very valuable. They're now thinking whether to adopt Clojure/Electric more broadly.#2024-02-2909:18mileloI'm trying to develop an application on electric and xtdb-in-a-box based on https://github.com/hyperfiddle/electric-starter-app. The server needs to serve two clients, a very simple one at the root that needs to load fast over a slow mobile connection and a potentially much larger client at "/admin" which will have good connectivity.
I've had a look at electric fiddle, I don't fully understand its routing yet, I'm guessing it behaves similar to a condition on a reactive value, however it seems the clients share the same runtime-code, an approach which wouldn't support my use-case.
It seems to me I need to split my code to have two distinct clients that compile separately against common server code and modify server-jetty.clj middleware to add a conventional router to serve these up with two electric-entrypoints and reactors.
Are there likely to be any issues with this? Are there any existing examples of such an approach? TIA#2024-02-2912:46Dustin Getztwo electric servers is not a tested configuration but i don't see why not#2024-03-0101:25VincentRealizing... might have to revive another earlier online forum project, now in Electric 😄#2024-03-0107:40braai engineerHTML forms in Electric seem to keep submitting after initial submit event failed and form is re-rendered or any atoms change. Happens with ui/input or plain dom/input with own change events. E.g.:
(e/defn Form []
(e/client
(let [!input (atom "text"), input (e/watch !input)]
(dom/form
(dom/on "submit" (e/fn [e]
(.preventDefault e)
;; this re-runs after initial failure if any fields change. is it because this re-runs when input changes? I think solution was to move transact! call into a Clojure fn last time.
(e/server (transact! input))))
(ui/input input (e/fn [v] (reset! !input v)))
(dom/button
(dom/props {:type "submit"})
(dom/text "Create Thing"))))))
#2024-03-0107:42braai engineerTried moving transact! into a clojure fn and same thing happens.#2024-03-0107:43braai engineerAlso tried moving the form submit handler to the submit button "click" event - same symptom.#2024-03-0107:50henrikTry e/snapshot input in the handler#2024-03-0107:50henrikOr otherwise just @!input#2024-03-0107:52braai engineerThanks, that fixes it! I understand that it "subscribes" to input, but seems weird that the "submit" handler doesn't run initially but then keeps running even though no submit event was fired, cc @U09K620SG.#2024-03-0107:53braai engineer@U06B8J0AJ e/snapshot does not seem to prevent it, but deref does#2024-03-0107:53henrikAh, interesting.#2024-03-0108:09braai engineerIn trying to fix the snapshot thing because I can't use the e/watch values without it looping, got caught by trying to deref the client-side atoms in e/server to get at the current value and saw a hard-to-debug "not concurrent future" error, so I spent a while trying to figure out where on server I was deref'ing on a Datomic tx.
Feels like Electric could know when you are dereferencing a client-side atom on server and send send the value over the wire? Dunno if it would be possible.
cc @U09K620SG
Have to do:
(e/client (let [data @!input] (e/server (transact! data))))
#2024-03-0108:34henrikThis is just a specific case of running a server-side function with data that sits client-side. Electric will make the data available to the function, not the function available to the data.
If the code states that you want the function (`deref` in this case) to run server-side, Electric would kind of be committing treason by arbitrarily deciding to run it client-side instead.#2024-03-0108:35henrikThe value of breaking this pattern for derefs in particular would not be worth the incongruity it would create IMO.#2024-03-0111:52Dustin GetzForms are the weakest part of Electric right now (ironic because we built it to solve that). Stay tuned, the electric-ui5 design promises to solve all these problems. In the meantime, you can use the workarounds, or also Henrik in his app uses out of band http post i believe#2024-03-0112:13henrikWe’ve managed to avoid side-channels with careful use of derefs and snapshots.#2024-03-0107:56braai engineerMaybe this is a dumb question, but is there a way to "reach into" or gather values up from a for-loop of components without callbacks, so that if you are rendering a bunch of field rows and you click a button in the container, all the values in the forms are gathered up for submitting to server? Maybe using point-effects "on the server"? (for lack of a better term) E.g.
(e/defn Input [v]
(e/client
(let [!state (atom v)]
(ui/input v (e/fn [v] (reset! !state v)
;; can this be done with a point-effect?
(ExposeToCallerSomehow.))))))
(e/defn Container []
(dom/div
(e/for [n [1 2 3]]
(Input. n))
(dom/button
(dom/on "click" (e/fn [e] (gather-state-from-Inputs-without-Callbacks))))))
I can get around this by passing in an Electric handler to each field that adds to a collection in the container on change, but would be convenient to use point-effects for this and gather up exceptions and other data.#2024-03-0108:37henrikYou could back all your form elements with a single state atom instead of individual state atoms, and then submit that single state when you click the submit button.#2024-03-0111:54Dustin GetzIt's a great idea, this callback-free pattern is what electric-ui5 will use, however our design required changes to Electric (e.g. the differential dataflow stuff). We intend to work on this after Electric v3 is delivered#2024-03-0316:45braai engineer@U09K620SG is v3 = incremental version?#2024-03-0316:46Dustin Getzno#2024-03-0316:45braai engineerWill incremental Electric change the DSL in any way?#2024-03-0316:46Dustin Getzdo you mean differential?#2024-03-0316:46braai engineeryeah#2024-03-0316:46braai engineerI say incremental because that is what issue #43 calls it: https://github.com/hyperfiddle/electric/issues/54#2024-03-0316:47Dustin GetzThat is something different, incremental compilation (fast compilation) shipped in December#2024-03-0316:48braai engineerhmm which version has incremental compilation? think I might still be on old version#2024-03-0316:48Dustin Getzsee changelog#2024-03-0316:50braai engineerelectric-starter-app currently uses SHA b32ac98df7d7ec87f225d47354671be172ffa87e which was updated on 25 Jan, so I assume that SHA has v540 or later 🙂#2024-03-0316:51Dustin Getzhttps://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#2024-03-0316:54braai engineerThanks, I did check the changelog and see that it landed in v536. Wasn't sure if that SHA was before or after fast compilation (I see that it is)#2024-03-0611:58simonkatzI’m using catch Pending to display a background image — it’s working fine.
Is there a way to do this only if the pending state lasts for longer than, say, 0.5 seconds, to avoid the UI “flashing” when actions complete quickly?#2024-03-0612:03xificurCmaybe https://github.com/hyperfiddle/electric/blob/0cfb98429f0e9b4cb7822173d55d1498ce2dd972/src/contrib/electric_contrib.cljc#L6#2024-03-0612:23simonkatzPerfect! Thank you#2024-03-0618:38danbuneaIn my quest about deploying options, I am looking at Amazon EC2:#2024-03-0618:39danbuneaSo the first option is to manually create an AWS EC2 instance, ssh into it, install git and docker and start it there#2024-03-0618:39danbuneaI put all the instructions in this readme: https://github.com/DanBunea/electric-starter-app#2024-03-0618:40danbuneaNext time, I will use ECS (Containers/Services/Task definition to get the docker image from ECR)#2024-03-0618:41danbuneafor now, if someone is a true newbie, this is how to set it up on EC2 manually#2024-03-0618:42danbunea#2024-03-0619:35Dustin GetzCool, and fwiw, the -67 is not a measure of latency, it's a measure of clock skew between client and server#2024-03-0621:08danbuneaI copied from the tutorial the TwoClocks component and put it in the main of the starter app#2024-03-0709:51Andrew WilcoxIs it possible to use plain dynamic binding with plain Clojure values on the server?
I'm keeping some state on the server for each connection. I might like to do something like:
#?(:clj (def ^:dynamic !client-state))
(e/def client-state)
(e/defn Main [ring-request]
(e/server
(binding [!client-state (atom {})]
(binding [client-state (e/watch !client-state)]
...)))))
This fails because !client-state isn't an e/def. Of course I can make it an e/def:
(e/def !client-state)
(e/def client-state)
(e/defn Main [ring-request]
(e/server
(binding [!client-state (atom {})]
(binding [client-state (e/watch !client-state)]
(UI.)))))
I'm not sure if this is a hack, or a natural way of lifting a Clojure value into the Hyperfiddle DSL using a constant reactive value 🙂#2024-03-0710:36henrikThat looks good to me. You can put multiple bindings in binding , so no need to nest them.#2024-03-0710:42Andrew WilcoxI don't know about the hyperfiddle DSL, but my understanding was that Clojure's binding was executed in parallel.#2024-03-0710:42henrikOh yeah, you’re right.#2024-03-0711:01henrikIf anyone is interested, we’re employing a React-inspired macro to declare state. E.g.,
(let [[current-count set-count!] (use-state 0)]
(dom/div (dom/text (str current-count)))
(dom/button
(dom/text "Increment")
(dom/on! "click" #(set-count! inc)))
(dom/button
(dom/text "Reset")
(dom/on! "click" #(set-count! 0))))#2024-03-0713:23Dustin GetzWhat do you think is the benefit over just using the native clojure atom ops?#2024-03-0713:30henrikThree minor things that just about add up to being worth it (in our opinion):
• Slightly more convenient to write, as the atom itself is inconsequential in 99% of cases: you want its value and a way to manipulate it.
• In the same vein, you have a setter that by virtue of the interface gets named closely to what it does. So, slightly cleaner semantics and a more direct way to get to them.
• The shape and function is familiar to React users, or users of UIx and other libs that follow the React form and function.#2024-03-0713:36henrikSimilarly, we have a $ macro as well (inspired by UIx).
So the above might look something like this in our codebase:
(let [[current-count set-count!] (use-state 0)]
($ :div ($ :text (str current-count))
($ Something {:hello :world})
($ :button {:class […]}
($ :text "Increment")
(dom/on! "click" #(set-count! inc)))
($ :button {:class […]}
($ :text "Reset")
(dom/on! "click" #(set-count! 0)))))
I.e., $ used for both instantiating reactive functions and dom elements, and to elide dom/props when the payload is a straight map (which it is most of the time).#2024-03-0713:37danbuneaI always have the atoms holding the state as more complex maps, and the changes that I do to them I always extract in separate functions.#2024-03-0803:30VincentHello, wondering about autofocus
(dom/props {:autofocus true}) ought do it?#2024-03-0807:05Geoffrey GaillardThis should work:
(dom/input
(.focus dom/node))
Autofocus is not trivial. See https://blog.maisie.ink/react-ref-autofocus/#2024-03-0816:35VincentThanks that worked perfectly#2024-03-0821:31VincentDudes, hyperfiddle is splendid.#2024-03-0902:02Jim DueyHowdy. I've been impressed with Electric (and Hyperfiddle before that) for a long time. But I'm just now starting to think about using it in anger. Is it possible to compile the server side of things to ClojuresScript rather than Clojure?#2024-03-0916:01Dustin Getznodejs server target is not a supported configuration, it is possible of course but needs a commercial use to justify (and fund maintenance)#2024-03-0919:39simonkatzAm I right in thinking that the following should throw Pending until my-db-query returns?
(e/server (e/offload #(my-db-query <args>)))
I’m seeing Pending thrown on initial load. When the values in <args> change, my-db-query is called but Pending is not thrown.#2024-03-0920:58Dustin Getzit depends- for pending to be thrown on the client, the client needs to be the origination of the change. in other words, if something changes on the server, the client is not aware of that and pending cannot be thrown#2024-03-0921:35simonkatzAh, maybe I understand…
Do I need to be calling the server within a callback on the client (as happens in the Chat demo)?
If I’m making my server call as a result of a watch on an atom (even if that’s on the client), then I won’t get pending?#2024-03-0922:59Dustin Getzthink of it this way- in a multiplayer chat app, if the message came from someone else’s tab, there can’t be a pending in your tab because your tab doesn’t know about the change yet. you’ll only learn about the new chat message when it arrives, at which point it’s too late for a Pending. you only see pending when the load is in response to your own event #2024-03-0923:07simonkatzOK, thanks, makes sense. I need to do a little refactoring. 🙂#2024-03-1010:37Geoffrey GaillardNote we have an experimental e/offload-task
• e/offload: throws Pending until the first result is available, further results are streamed in without seeing further Pendings. Pending -> value -> value -> value -> ....
• e/offload-task: throws Pending until first result is available. Then throw Pending again while the task re-runs. Pending -> value -> Pending -> value -> ....#2024-03-1011:35simonkatz> Note we have an experimental e/offload-task
Thanks — that seems to do exactly what I want.#2024-03-1011:39simonkatz(I have a watched client-side atom for a user input; now when the user edits the value and I make a server call as a result of the DAG reactivity, I get Pending.)#2024-03-1002:56Andrew WilcoxElectric is pretty amazing.
I have a calculation that I'm doing on the server, a purely functional transformation of the function input to the function output. While I'm developing the function, I'm changing the code of the function, so the output depends on both the input and the current code defining the function.
Normally these would be separate implementations. Usually the program would be written expecting that the function definition isn't going to change, and to rerun the function if the input changes. If I changed the code, then either I'd run the program again, or reload the code, or have the code reloaded for me through a hot code reload feature.
I already had the input in an atom, referenced using the usual (e/def input (e/server (e/watch !input))). I set up a file system watch so I'd get a callback when the source code file changed. I use Clojure's (import 'my-namespace :reload) to load the new code when the source file changes, and then store the updated function in its own atom !calc-fn
Next was the question of how to get this wired into Electric. I poured myself a fresh cup of coffee, and prepared to do a deep dive into the Electric examples and notes, and perhaps struggle my way through the Missionary documentation to see if I could find an answer.
(e/def calc-fn (e/server (e/watch !calc-fn)))
(e/defn UI []
(e/client
... (e/server (calc-fn input)) ...
And that was it.
The call has reactive dependencies on both the calculation function and the calculation input, so if either changes the output is updated. I can change the source code of the calculation, press save, and the output is updated instantly.
I implemented by own custom hot code reload feature in less time than it probably would have taken me to go find and figure out how to use a hot code reload library.#2024-03-1104:30Vincentserver-client network transparency is rad and a boon / very helpful.#2024-03-1009:29joshcho2i have this code and would like to block the img from showing up until the url is downloaded. how would i do that?#2024-03-1010:50Geoffrey GaillardThe {:tag :img, :attrs nil, :content nil} element has a load event (https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/load_event)
This should work (haven’t tried it)
(dom/img
(dom/props {:src ""
:style {:width "256px"
:height "256px"
:display :block
:margin-top "20px"
:visibility :hidden} ; hidden by default
:class "cloud-image"})
(when (dom/on! "load" (constantly true))
(dom/props {:style {:visibility :visible}})))
Note some browsers might not load the image if it is not visible.
(visibility: hidden, display: none, etc.).
On the other hand, a better approach could be to give your img a background placeholder, which the browser will happily render while your image is loading.
You can do so with regular CSS.#2024-03-1011:57joshcho2thank you so much!#2024-03-1011:57joshcho2i'll definitely try this#2024-03-3111:36braai engineeris there a difference between dom/on and dom/on!?#2024-03-3112:15xificurCOne takes an e/fn, other cc/fn#2024-03-1119:12TommyI have a missionary stream (not signal) coming from an http request, and i want to see those values in the dom/collect them my own logic. right now I have
(e/defn Response [v]
(let [r (e/server (new (llm/ask-ant-stream v)))]
(e/client
(dom/h4 (dom/text r)))))
and its only rendering the very last value in the stream. When I do this, it prints the values in as they come in
(m/? (m/reduce conj (m/ap
(println (m/?> (ask-ant-stream "hello"))))))
any ideas? am i misunderstanding and missionary streams are treated as signals in electric? what is the best way to do what I am trying to do (show every value of stream onto dom) maybe something like
(e/client (m/? (m/ap (reset! !val (m/?> (e/server (llm/ask-ant-stream "hello")))))))
#2024-03-1120:37Dustin Getzyou can use m/reductions to produce a discrete succession of accumulated values, which electric will consume as a continuous flow#2024-03-1121:28Tommywhen I do that, I still am only seeing the very last item in the stream of reductions#2024-03-1300:13Tommythis worked, thank you#2024-03-1310:28bdbrodieThanks for getting this working in public, very relevant case these days.#2024-03-1411:14simonkatzMy app has a glitch. I have a list of data items, a search string input and a search status display (blank input / found / not found). The search status has a background color to indicate the state, and it briefly but noticeably flashes a wrong color when the user changes the search string from empty to “1” (for example).
I have a minimal repro — see details in my in-thread reply.#2024-03-1411:15simonkatz#2024-03-1411:17simonkatzIf the input is an integer between 1 and 100, that’s regarded as :found. An empty input is :blank, and the other alternative is :not-found.
When the user changes the input from an empty string to “1”, I get the following output (with an unwanted :not-found that is causing my glitch):
---------
:not-found "1"
search-str = "1"
:found "1"#2024-03-1411:17simonkatzI can make the glitch go away if I do any of the following:
• move the body of FoundStatus to the call site
• wrap the call of FoundStatus in (e/server ...)
• remove the (e/client ...) wrapping of the call of FoundStatus.#2024-03-1411:18simonkatzIn my actual app I haven’t found a simple workaround that works, and in any case there’s the possibility that when I refactor I would re-introduce the glitch.#2024-03-1411:18simonkatzI guess this is a bug, but is there something I can do to work around this? Maybe a way of saying to only pass on a reactive value if it’s had the same value for a certain amount of time?#2024-03-1412:53Dustin GetzI think this is what we call the "when true" bug, I will need to look more closely to be certain#2024-03-1416:14simonkatzI said:
> In my actual app I haven’t found a simple workaround that works
FWIW, that’s not true. I’ve wrapped the call of FoundStatus in (e/server ...) and all is good.#2024-03-1416:36Dustin Getzok#2024-03-1416:36Dustin Getzone trick that often helps, is this idiom: (case found? (FoundStatus. search-str found?)) – the case forces found? to resolve (not be Pending) before evaluating the body. I'll need to dig into our issue tracker to remember the exact details of the bug#2024-03-1416:39Dustin Getzto be clear, if the bug is what i think it is, it should just be fixed, and i dont know if the (case x ...) pattern has real world use once Electric is perfected. This particular bug won't be fixed until Electric v3, it was a deep issue#2024-03-1416:50simonkatzThanks for the suggestion and additional details.
I’ve tried (case found? (FoundStatus. search-str found?)), but it makes no difference. 😞#2024-03-1414:43awb99I think hyperfiddle could benefit by having an options-ui. The idea by options-ui is to declarative specify which options should be collected from the user and which choices are allowed.#2024-03-1415:07Dustin Getzthis sounds a bit like HFQL, which is a powerful hypermedia DSL described on the new website - https://www.hyperfiddle.net/#2024-03-1415:09Dustin Getzin fact the reason we built Electric was to make HFQL possible#2024-03-1417:10awb99Is there a repo for this?#2024-03-1417:11Dustin Getznot public yet#2024-03-1419:00awb99I am interested to see it!#2024-03-1414:43awb99https://github.com/pink-gorilla/options/blob/main/demo/src/demo/page/trateg.cljs#2024-03-1414:48awb99The benfit of that is that one can have different ui elements (textbox; combobox; date-picker; button; user defined elements) and so this options then can be used to make requests to clj or just to render ui differently.#2024-03-1414:52awb99Another idea how electric could become easier to use is to bring in Tlthe concept of extensions. An extension is something that adds features to electric without having to do any configuration. The idea is that once you add an extension to the classpath by having it included as a dependency then it will automatically add it to the cljs build. This makes it a lot easier to reuse frontend ui components. An example is here: https://github.com/pink-gorilla/ui-highcharts/blob/master/resources/ext/ui-highcharts.edn#2024-03-1414:54awb99So having a resource that matches #"ext/.*/.edn" it woudl read the content: {:name "ui-highcharts"
:cljs-namespace [ui.highcharts]} and in this case it would add ui.highcharts to the shadow build.#2024-03-1417:32Kurt Harrigerdoes anyone have css hot reloading working?#2024-03-1417:33Dustin Getz.css artifacts hot reload via shadow which i believe works out of the box in the starter app, what are the specifics of your setup?#2024-03-1417:36Kurt HarrigerI was just trying to throw css into the index page and see if it would hot reload but doesn’t seem to be. I just forked the starter app and tried similarly and its not working.. I’ll push a branch really qucik#2024-03-1417:39Kurt HarrigerThis is what I tried to add to starter app https://github.com/kurtharriger/electric-clojure-state-issue/commit/6c3269ba4627cb13c0be1454a134aa41adf4874b#2024-03-1417:41mgxmtry to change :http-root to :watch-dir#2024-03-1417:42Dustin Getzhttps://github.com/hyperfiddle/electric-fiddle/blob/main/shadow-cljs.edn#L6#2024-03-1417:42Dustin Getzyou're right, the starter app may be broken, i'll log a ticket#2024-03-1417:44Kurt Harrigerstarter app didn’t include any css so not really broken but an example would help#2024-03-1417:45Dustin Getzok#2024-03-1417:47Kurt Harrigerchanging to watch-dir seems to have fixed it#2024-03-1417:47Kurt HarrigerI’ll send PR#2024-03-1417:55Kurt Harrigerhttps://github.com/hyperfiddle/electric-starter-app/pull/19#2024-03-1505:17Andrew WilcoxThank you for this!#2024-03-1418:35chaosHi, in the below program, there is a reactive input that starts at directory c:/tmp/ which has 5 files as returned from the server. There are two println forms that print the current dir and cnt of files to both server and client respectively the both print c:/tmp/ and 5 count of files at startup as expected (the c:/tmp/ directory has 5 files in it). When the user appends the letter e to the input field (i.e. c:/tmp/e ) the server looks at that c:/tmp/e directory and returns 0 files (the directory is empty). So the expected output of the program when this happens should be c:/tmp/e and 0 files for both server and client, though on the client side I'm getting a print out of c:/tmp/e dir and 5 files first, and then the expected c:/tmp/e and 0 files. I was expecting the client println form to only fire once with c:/tmp/e and 0.Why is that so?
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
;; Saving this file will automatically recompile and update in your browser
#?(:clj
(defn dir-list [dir]
(.listFiles ( dir))))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(let [!dir (atom "c:/tmp/")
dir (e/watch !dir)]
(ui/input dir (e/fn [v] (reset! !dir v))
(dom/props {:type "search"}))
(e/server
(let [files (dir-list dir)
cnt (count files)]
(println :dir dir :files cnt)
(e/client
(println :dir dir :files cnt))))))))
;; server output
;; :dir c:/tmp/ :files 5
;; :dir c:/tmp/e :files 0
;; client output
;; shadow-cljs: #3 ready!
;; :dir c:/tmp/ :files 5
;; :dir c:/tmp/e :files 5 ;; didn't expect this
;; :dir c:/tmp/e :files 0
more details at https://github.com/hyperfiddle/electric/issues/68#2024-03-1418:55Dustin GetzHi - thanks#2024-03-1418:55Dustin GetzWhat seems to be happening here is you expect the dir watch to be transactional with the cnt derived value#2024-03-1418:57Dustin Getzthe dir watch is on the client, and cnt is on the server, so we're seeing latency, but you reasonably expect (println :dir dir :files cnt) to only run after all parameters are available#2024-03-1418:59chaosright, I was expecting the same semantics as if I was reading a plain clojure without the distiction of client/server (apart from the atom reactiveness)#2024-03-1419:00chaosor put it perhaps differently, I was expecting the value of dir and cnt to be synchronized, given that cnt is dependent of dir#2024-03-1419:03Dustin GetzI need to confirm with the team (Electric v3 is sharpening a bunch of stuff). Broadly the consistency model is such that client-internal propagations result in consistent states as guaranteed by missionary, and same for server-internal propagations, but for distributed propagations there are some tradeoffs, but it does IIUC seem like the Pending cnt and the dir should be known simultaneously and therefore the println should not ever see that inconsistent state#2024-03-1419:04Dustin GetzLet us discuss internally and get back to you#2024-03-1510:00henrikSee what would happen if you enforce a synchonization point before the print:
#?(:clj (defn search
[dir]
(let [files (dir-list dir)]
{:dir dir
:files files
:cnt (count files)})))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(let [!dir (atom "c:/tmp/")
dir (e/watch !dir)]
(ui/input dir (e/fn [v] (reset! !dir v))
(dom/props {:type "search"}))
(e/server
(let [{:keys [dir cnt]} (search dir)]
(println :dir dir :files cnt)
(e/client
(println :dir dir :files cnt))))))))
In this case, I think you should see consistent output, since the Clojure function would ensure that the count changes as the dir changes, rather than independently of each other.#2024-03-1516:34Dustin Getz@U012BL6D9GE I spoke with the team. We believe this is exactly a known and previously resolved issue we call the "distributed glitch", which is: when cnt should be pending because it is derived from dir via the network. Electric v2 already accounts for this circumstance and therefore the behavior you observe is unexpected and we have logged it as a bug.#2024-03-1520:25chaosThanks @U09K620SG for looking into this, and I can confirm @U06B8J0AJ workaround should also work (I've tried something similar in my code at the time).
Just trying to set my expectations on the bindings resolution across boundaries, because it caught me by surprise.
In a even more simplified example below, I was similarly expecting the program to be read from top to bottom like a "normal" clojure program, with electric taking care of efficiently transporting the values under the hood as need. If a reactive change happens at the higher form, the flow continues synchronously to the inner forms where the variables are affected.
The program starts by sleeping for 100 ms amount of time on the server, and then reports on it from both sides. When the button is pressed, the sleep amount is increased by 1, and the reactive process is triggered once more.
There's an extra reporting on the client, that says that ms is 101 but the server-msg is slept for 100 ms, which breaks my expectations.
If I am not to reason about the program as one that the flow of information is sequential and bindings dependencies are maintained in the flow across the boundaries, how should I expect the reactive update part to behave? Am I responsible forcing synchronization of the bindings across boundaries? I would have thought electric has all the information required to do so in the graph but it might as well be a hard problem to solve automatically, given the nature of the setup with n-clients communicating with a single server.
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
;; Saving this file will automatically recompile and update in your browser
#?(:clj
(defn sleep [ms]
(Thread/sleep ms)
[:slept-for-ms ms]))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(let [!ms (atom 100)
ms (e/watch !ms)]
(ui/button (e/fn []
(println :client :sleep-increase)
(swap! !ms inc))
(dom/text "inc sleep"))
(println :client :sleeping-for ms)
(e/server
(let [server-msg (sleep ms)]
(println :server :sleep-for-ms ms :server-msg server-msg)
(e/client
(println :client :sleep-for-ms ms :server-msg server-msg))))))))
;; client output
;; => Connecting...
;; => shadow-cljs: #7 ready!
;; => Connected.
;; => :client :sleeping-for 100
;; => :client :sleep-for-ms 100 :server-msg [:slept-for-ms 100]
;; button pressed
;; => :client :sleep-increase
;; => :client :sleep-for-ms 101 :server-msg [:slept-for-ms 100] ;; I wasn't expecting this to trigger
;; => :client :sleeping-for 101
;; => :client :sleep-for-ms 101 :server-msg [:slept-for-ms 101]
;; server output
;; => :server :sleep-for-ms 100 :server-msg [:slept-for-ms 100]
;; button pressed
;; => :server :sleep-for-ms 101 :server-msg [:slept-for-ms 101]
#2024-03-1520:28Dustin GetzI don't understand your question, that is a lot to unpack#2024-03-1520:30Dustin GetzElectric maximizes concurrency, it is not sequential/imperative evaluation#2024-03-1520:31Dustin Getzthe line you commented looks like the same glitch as your original report, it is a bug#2024-03-1520:32Dustin Getzthe sleep function is also wrong because it is synchronous blocking, it needs to be an async missionary sleep#2024-03-1520:39chaosYeah, sorry not being clear. I'm just trying to understand how to reason about the program. My expectation was that I can read the program the "same" as a clojure program, and that is sequentially, in which case that line that I mentioned above shouldn't have happened. If this is also due to the "glitch" then I am covered, the program can be reasoned by me as if it was sequeantal then and this is how I expect it to run. I thought the "glitch" described earlier, was when trying to send two bindings across from the server, in this case is only one.#2024-03-1520:41chaosThe use of sleep is just for demonstration purposes, I just wanted to have a a server funtion that will take "some" time to complete (i.e. doing I/O), and sleep was the easiest to use to make my point#2024-03-1520:48chaoswhat I mean by sequental evaluation in the presence of reactivity is how the various forms are triggered.
So when the program starts, I expect the following "sequential" evaluation of bindings: !ms->ms->(println :client :sleeping-for ms)->(sleep ms)->server-msg->server println->client println, which does happen
I would have expected the same flow to happen after the button is pressed, but I am getting the extra print out.
And thus the question.
Though now you have said that this is the same bug as before, I'm covered.#2024-03-1520:49chaosBtw, is there a way for me to track the issue, maybe as a github issue, or shall I periodically look at master checkins?#2024-03-1520:49Dustin GetzThat sounds right, the evaluation order should exactly match the dataflow DAG, effects run when one or more inputs change and you should never see inconsistent states#2024-03-1520:52Dustin GetzWith respect to the issue, we've added it to our internal tracker and will look when we have time#2024-03-1509:59euccastroAre you supposed to be able to build an uberjar while the app is running in the dev environment?
If I run /usr/local/bin/clojure -X:build:prod:fazenda uberjar :hyperfiddle/domain fazenda :build/jar-name app.jar while the dev app is running I get
Execution error (ExceptionInfo) at shadow.cljs.devtools.server/check-for-other-instance! (server.clj:415).
shadow-cljs already running in project on . Use or terminate it before starting another one.
#2024-03-1510:02euccastroIf I restart the dev app including the build and prod aliases and I run the equivalent
(build/uberjar {:build/jar-name "app2.jar" :hyperfiddle/domain "fazenda"})
Then the uberjar gets built (I haven't tested it yet, though), but the running web server is killed:
user> (build/uberjar {:build/jar-name "app2.jar" :hyperfiddle/domain "fazenda"})
INFO build: build/uberjar {:build/jar-name "app2.jar", :hyperfiddle/domain "fazenda"}
[:prod] Compiling ...
[:prod] Build completed. (189 files, 0 compiled, 0 warnings, 17.83s)
INFO io.undertow: stopping server: Undertow - 2.3.10.Final
Worker shutdown.
INFO build: fazenda client built
INFO build: build/uberjar included aliases: [:prod :fazenda]
INFO build: app2.jar
nil
#2024-03-1510:08euccastroI see that the :dev and :prod shadow-cljs configurations have the same :output-dir, so it doesn't seem like building prod jars while the shadow watch is running is an expected use case. Would it be hard to change that in the electric fiddle project? Are patches accepted for this?#2024-03-1512:04Dustin GetzI doubt we ever tried this, if you're confident in the solution sure we are likely to merge a PR#2024-03-1517:12VincentBenefits to using Hyperfiddle Electric:
+ General clojure benefits (sanity levels reach all-time highs when working on long-term projects)
+ mock-ups become reality and if you're careful with your data model, mock-ups go very far if not all the way to prod frequently#2024-03-1622:28Nikolas PafitisHello guys. I'm trying electric with different databases. In your experience, what database makes it easier to work with in electric, particularly when creating a flow from a database for 'live' queries. I know of this https://github.com/hyperfiddle/electric-fiddle/blob/28b875d3eaa77f2d8b4716f383ba8f5ed2e1b92e/src/xtdb_demo/xtdb_contrib.clj with XTDB although it only works when xt node is in-process. What other options are there?#2024-03-1622:36Dustin Getzwe have personally only used datomic using the tx-report-queue in the simplest unoptimized configuration#2024-03-1622:38Dustin Getzdatabase usage (including crud patterns) likely going to be a major work area for us in the back half of this year, it’s a weak point today #2024-03-1622:42Dustin Getzbroadly it is the database/data layer responsibility to inform electric via events when queries have changed - see rama, materialized for data layers that natively support query subscriptions. for batch databases, there is always going to be some impedance at this boundary. missionary gives you the tools you need to manage it. you can always fallback to SPA/REST semantics where queries are polled on navigate for the most part except for specific realtime views (e.g. chat views) which likely aren’t going to be stored in postgres anyway, you’d use an event store which offers native topic subscriptions #2024-03-1710:03Nikolas PafitisThanks for the answer @U09K620SG#2024-03-1710:24Nikolas PafitisHas anyone tried out surrealdb and live queries?#2024-03-1710:25Nikolas PafitisI guess it could be made to work nicely with missionary flows#2024-03-1702:52Andrew WilcoxBy the way, I found this old documentation on Electric Missionary interop helpful: https://github.com/hyperfiddle/electric/blob/b9bcfab1c81ca1815196af7b39d6f460d6d1cdc5/src-docs/user/electric/electric_missionary_interop.clj#2024-03-1722:28VincentFriends, ladies, gentlemen, wookies (Chewbaca clearly Clojurista), and the variety of code wizards/witches/ninjas,
Have created a GPT for Electric Clojure: https://chat.openai.com/g/g-yRbCuc8cz-electric-clojure-hypermind.
Membank is seeded with examples and the init prompt reminds it of the most suitable approach ("stop reaching for reagent, pls, it's e/defn thx").
https://chat.openai.com/g/g-yRbCuc8cz-electric-clojure-hypermind
Let me know where it sucks so I can enhance it. Nurturing my psychic bond with the GPT, so be nice 😅#2024-03-1800:52tobiasThis looks great, thanks for creating it! Something I’ve been missing with electric is the ease of the “paste HTML as hiccup” when I work with reagent in Cursive. Looks like with your assistant I can paste HTML and get electric dom code ⚡ #2024-03-1801:09Vincent@UA9399DFZ that sounds helpful, i think it will do that much. It still does not comprehend electric 100% because it wants to "subscribe to watch" things and "notify all peers" when swaps happen, so it doesn't really grok it quite yet, need to give it some more info to give it the electripiphany#2024-03-1801:28VincentAdding the files that give ChatGPT the insight into the macros (e/def, e/defn, and showing it and telling it about the masking of (let...))) Has proven to be very instrumental in getting the model to rely on correct embeddings (much like a pianist stayn in key)
Imperfectly perfect, I'm sniffing out shortcomings by asking it to generate novel examples. "Build a multiplayer web app" is a great way to see if it comprehends what's what.#2024-03-1804:07VincentThe intro prompt can only be 8000 chars and you can only add 20 files to the membank. But I'm thinking maybe I can cram all the examples into one big file#2024-03-1814:51grounded_sageI find this less useful than my own prompting against base ChatGPT.#2024-03-1814:51grounded_sageNot to dismiss the work going into this. I also want an Electric savvy GPT. I'm just reporting my experience.#2024-03-1816:10Lidor Cohen@U05095F2K I imagine examples might help @U055PQH9R4M get the most out of your experience report 🙂#2024-03-1816:34VincentI am ceaselessly tweaking it at the moment, I have found that giving it straight-up macro definitions can help. How are you prompting your GPT4, @U05095F2K?#2024-03-1816:53grounded_sageI freestyle the prompts most of the time. But it generally starts by generating a bunch of meaning dense context and encouraging the encoding of meaning and context into emojis when it feels it is appropriate to generate better responses. Negating patterns of expression that give less intelligent responses eg lists, introductions and conclusions. Enhancing expressions that are more intelligent.#2024-03-1817:12Vincent@U05095F2K Not gonna lie, I comprehended maybe 31% of what you said, but it sounds like you selectively lambast your GPT into excellence. what have you tried with this one in this thread and what results would you want instead of what you got?#2024-03-1821:12VincentHave made some enhancements, let me know if it is still insane. Sometimes it doesn't believe Electric is a real fullstack framework lmao. Not sure what is the correct cocktail of proof#2024-03-1823:54joshchoi’d experiment with claude for this btw#2024-03-1823:54joshchoclaude opus >> gpt4#2024-03-1900:35Vincent@U6D1FU8F2 was just thinking to schedule a call with you ) and also, do you recommend Claude Pro#2024-03-1900:35Vincent@U6D1FU8F2 plz check out what's been cooking ^, i think you will be pleasantly surprised if you pick one of the 3 default prompts)#2024-03-1900:39joshchoyou need to use opus bc opus is really good so yes claude pro#2024-03-1900:39joshchowill do#2024-03-1911:24siddharth yadavHi @U055PQH9R4M, is hyperfiddle.electric-svg ns added in the mem bank? I was trying to convert html svg to electric svg, the returned code used dom/svg when asked it said there is no hyperfiddle.electric-svg ns, apart from this everything was correct.#2024-03-1911:30siddharth yadavAlso are we following any naming conventions? It seems to be PascalCase I was expecting kebab-case#2024-03-1912:24siddharth yadavby default it uses (e/atom ..) to create a new atom#2024-03-1913:41Vincent@U049Q2LP68M thanks I had the svg example in there for a while but removed it, will backplace.
+ kebab-case, heard.
+ yeah it thinks e/atom is a thing. Ok thanks for those ... applying fixes#2024-03-1913:49Vincent@U049Q2LP68M
check out https://chat.openai.com/share/475f09c5-1ebe-494a-9e61-c6b47d5739c6
"Let's make multiplayer game using dom/svg."
Looking better?#2024-03-1917:08siddharth yadav@U055PQH9R4M In your chat, the gpt used dom/circle instead of svg/circle , I thought this could be fixed if I only said svg instead of dom/svg so I tried the prompt Let's make multiplayer game using svg. but I still got dom/circle . Adding the instruction for svg components only use svg namespace
fixed the issue and it correctly used svg/..
In the following chat it got svg/circle right but had other bugs
https://chat.openai.com/share/6bc73a93-8789-4e06-9026-a38d04145ff3
• (dom/button (dom/props {:on-click (e/fn [] .. instead of using (dom/on! ...
• dom/on vs dom/on! #2024-03-1917:30VincentOh is the canonical actually dom/on! ? 😅
yeah the GPT loves to do :on-click, would be good to coax it out of this "habit"
thanks for adding svg components only use svg namespace#2024-03-1917:42VincentMajor update to the initialization prompt this morning. Have fun testing out le hypermind#2024-06-0215:27grounded_sage@U055PQH9R4M I played around with creating a GPT today. But focused on Missionary.
https://clojurians.slack.com/archives/CL85MBPEF/p1717341546748089#2024-06-0215:29grounded_sageAlso I have not tried this one. I could play it a bit. I find it way better still to create mega prompts self created by them. The instruction character count limit is very ... limiting.#2024-03-1815:48nakkayaI am probably missing something obvious but here is the stripped down version of what I am trying to achieve,
(let [record (e/server
(let [u (e/client record-id)]
(e/offload #(query-record u))))
!disabled (atom true)
disabled (e/watch !disabled)]
(if (= disabled true)
(dom/div
(dom/button
(dom/text "Edit")
(h/on-click
(reset! !disabled false)))
(RecordForm. record disabled))
(dom/div
(dom/div
(dom/button
(dom/text "Cancel")
(h/on-click
(reset! !disabled true)))
(dom/button
(dom/text "Save")
(h/on-click
(e/server (println "on-click" (e/client record)))
(reset! !disabled true))))
(RecordForm. record disabled))))
RecordForm takes a record map and shows a form, when disabled is true all text fields are shown deactivated when it is set to false form is editable and a set buttons are used to toggle the state. All is working except the part where I try to print the record itself, all fields of the record are nil however UI has all the correct values shown. i.e record has a first-name field it has the correct value in the UI but when on-click runs it's value is nil .#2024-03-1819:35VincentCan you do this with (ui/button (e/fn [] (e/client ...))) please#2024-03-1819:36Vincent(ui/button
(e/fn []
(e/client
(reset! !disabled false)))
(dom/props {:class "ayyy"})
(dom/text "Reset the !disabled atom to falesenesesess"))#2024-03-1820:08VincentIt's important to use the ui4 elements such as ui/button dom/input since they are macros that will be redefined during the Electric compilation process.#2024-03-1820:27Dustin Getz@U5H4U2HEH hard to say without an actual reproduction#2024-03-1820:39nakkaya@U055PQH9R4M I get the same output. @U09K620SG I'll try to create a minimal working example. One more questions, under what conditions we should pick ui/button over dom/button ?#2024-03-1820:46Dustin Getzui4 namespace is intended to be more abstract with no access to the underlying event (iirc), it is also experimental#2024-03-1820:46Dustin Getzdom namespace maps directly to underlying dom api without abstraction#2024-03-1820:48nakkaya👍 thanks#2024-03-1820:55Vincent@U5H4U2HEH if print is the only thing not working, it is something there.
I think you have record wrapped with e/client but it is an e/server value 😉#2024-03-1822:36nakkaya@U09K620SG @U055PQH9R4M I think I've figured out whats going on. A record has the following structure {:a #object[cljs.core.Atom {:val :b}]} if I do not deref values before using it on the server side it gets transfered/printed as {:a nil} .
(e/server (println "on-click" (e/client record))) ;; does not work as expected
(e/server (println "on-click" (e/client (pr-str patient)))) ;; prints as expected
(let [patient (e/client (update-vals patient deref))] ;; works as exptected
(e/server (println "after-deref" patient)))
#2024-03-1822:48VincentYes, when passing between client and server some values are "serializable" and others not. However, the serverside executable ought mention that you are doing an unserializable reference transfer` or something...#2024-03-1822:53Dustin GetzV is right, the server should log when unserializable reference transfer is attempted#2024-03-1822:54Dustin Getzwith electric it is best to only use atoms unnested, not in records or collections#2024-03-1822:57nakkayaunserializable reference transfer is attempted
I used to get that message once in a while but in this case it did not get printed.#2024-03-1823:14Dustin Getzit is almost certainly either a logger misconfiguration or an editor issue#2024-03-1900:36VincentOh is record a clojure thing via defrecord and therefore needs explicit support perhax?#2024-03-1820:49VincentAnyone with an uberjar deployed, what's your mem usage?#2024-03-1820:55Dustin Getztutorial app 30 day view#2024-03-1820:55Dustin Getzdatomic viewer 30 day view#2024-03-1821:11VincentBrilliant, thank you#2024-03-1821:26VincentDesign question, for someone making a seek bar for audio playback 🎛️ 🎼
Does it make sense to use (dom/range ...) or is it perhaps better to invent some sort of canvas component to capture clicks? Looking for the least effort maximum payout 80/20 here, but also usable#2024-03-1916:33Dustin Getzthere may be third party js widgets you can integrate#2024-03-1917:31Vincentoh nice idea.#2024-03-1917:42VincentHelp. What is the difference
dom/on
dom/on!#2024-03-1917:43Vincentdon't say dom/on factorial#2024-03-1917:47Kurt HarrigerI prefer to call it bang rather factorial#2024-03-1917:51noonianThe main difference is one takes a cljs callback fn and one takes an electric fn. There are more subtle differences also#2024-03-1917:51Kurt Harrigersource indicates maybe on! should be used when there are side-effects, but I’m not super clear on it#2024-03-1917:55noonianon! Takes a cljs fn and should work as expected in browser. on takes an electric fn (called with new) and terminates on the first non-pendijg value it returns (I believe). There can be issues if your electric fn does stuff that would execute expressions multiple times because the callback "lifecycle" does not survive past the first returned value#2024-03-1918:14VincentThanks @U052PH695 so if we're using raw cljs callbacks like (reset! le-atom new-val) dom/on! is the way to go#2024-03-1918:25noonianYes, that should work unless the atom was on the server :)#2024-03-1918:35VincentAha gotcha. Very good very good#2024-03-2008:13henrikI’m using an atom client side to contain a paginated list of items. When it loads, it contains 10 items on page 1. When the user navigates to page 2, 10 more items are loaded and added, and so on. This means that were they to go back to page 1, those 10 items are available immediately.
This is conditional of a category selection. So all items loaded are in a specific category. This means that if the user switches category, the cache atom now contains all items from the previous category, plus 10 new items. It’s a side effect of being stateful, and a classic and major downside of doing it this way.
What happens currently is that all items not belonging to the current category is cleared out of the atom on switch, and this works. In React, the same problem would probably be solved by giving the root component a key based on the current category, which would cause React to tear down the current state and rebuild it, thus effectively masking the statefulness happening in it.
Not that I want to use it, but is there a mechanism like this in Electric? Can you make an arbitrary tree conditional on some hash?#2024-03-2009:44Dustin Getzfor-by can be wrapped i.e. put a single child branch in a collection size 1 and manage it with a key#2024-03-2009:45Dustin Getzi also needed this at least once, maybe it should be made public api #2024-03-2009:48henrikAh, that’s clever. It feels like something one shouldn’t make a habit out of. But for clearing out e.g. states holding settings etc. that specifically controls views, and doesn’t really rely on data streamed from the DB etc., it could be an acceptable pattern to reset to defaults reliably.#2024-03-2009:51henrikA function that wraps it could be convenient.#2024-03-2010:05henrikAn attempt:
(defmacro given
[key & body]
`(first (e/for-by identity [_k# [~key]]
~@body)))
(given something
(dom/div (dom/text "Hello"))
(e/on-unmount #(println "Torn down")))
#2024-03-2010:06henrikNot sure would it should be called. But theoretically then, given a change in something , “Torn down” should be printed though there are no dependencies.#2024-03-2013:39henrikThis works as expected. I ended up rewriting it as,
(defmacro while
"Given a truthy `key`, executes the reactive `body`.
`body` re-executes for each distinct value of `key`."
[key & body]
`(first (e/for-by identity [k# [~key]] (when k# ~@body))))
It’s semantically tenuous, but clojure.core/while felt like the closest thing to map the behavior to.#2024-03-2018:57VincentI enjoy your macro and wonder about the perfect name#2024-03-2108:50henrikThanks! I guess it’s more Dustin’s while than mine though, since he suggested the core mechanic.
It’s interesting, for example
(e/defn TestWhile
[]
(e/client
(let [!a (atom 10)
a (e/watch !a)]
(while (and (pos? a) a)
(println "Hello")
(swap! !a dec)))))
If this while is cljs.core/while, this prints “Hello” ~100 times and then crashes the app.
If it is the while above, it prints “Hello” 10 times and then stops, with no crash.
It makes sense why the core while crashes the app, but incidentally the bespoke while exhibits behavior closer to how while works outside of Electric.#2024-03-2112:30Dustin Getzwhat is the crash from core/while?#2024-03-2112:31henrikI didn’t pay attention to the exact message, let me have a look#2024-03-2112:34henrikOn the client, Websocket is already in CLOSING or CLOSED state .#2024-03-2112:34henrikOn the server#2024-03-2112:35henrikBut I’m basically DOSing myself, so this isn’t super surprising IMO#2024-03-2112:35Dustin Getzagree it's not surprising, but it still might be a bug that it throws NPE (though certainly the behavior isn't defined)#2024-03-2112:35henrikAh, right#2024-03-2113:05Vincentdo-when is what I want to call it, in certain contexts#2024-03-2202:16Andrew WilcoxVery cool. I think that the name while doesn't quite capture the idea: the important functionality this provides is to remount the body when the key changes.
I suggest perhaps distinguishing#2024-03-2011:04mileloI've created an https://github.com/milelo/electric-multi-client-app, a minimal Electric Clojure app based on https://github.com/hyperfiddle/electric-starter-app providing a server with multiple independent clients, routing and an https://v1-docs.xtdb.com/guides/in-a-box/ database. Each client has its own dependencies and electric reactor.#2024-03-2018:42braai engineerHow should I go about securing my Electric routes while exposing some public routes on the same server?
In the starter app template, the recommended place to inject (wrap-authentication) like basic auth is in electric-websocket-middleware, which fires before anything else, returning 401 for all routes.
Is it possible to add auth middleware only to Electric websocket upgrades?#2024-03-2018:57Dustin GetzHave you considered using an if statement at the top of your electric app?#2024-03-2018:57braai engineerI want to exclude auth from some non-Electric routes, e.g. /api/do-thing, and /public-thing.html#2024-03-2018:57braai engineerOtherwise I would do it all in Electric#2024-03-2018:58braai engineerDeploying two different apps on two different domains or ports is an option, but then I have to deploy two things.#2024-03-2018:59Dustin GetzOh ok, that seems like a ring question, surely it is possible, i would find the ring support channels and ask there#2024-03-2018:59braai engineeryeah I guess my question becomes: can I safely move the electric websocket detection to later in the middleware pipeline, or will content-type stuff mess with it?#2024-03-2022:49nakkayaIf you search my messages I should have posted examples on this. In my case electric app lives in /app and authenticated and rest of the routes are unauthenticated.#2024-03-2109:32mileloYou could secure it externally (for free) through https://www.cloudflare.com/en-gb/products/tunnel/#2024-03-2110:11hkjelsHas anyone created hooks for clj-kondo to prevent the warnings with e/server & e/client?#2024-03-2112:29Dustin Getzi believe not, but as of the recent incremental compilation changes that released around christmas, this may now be feasible, namely because platform reader conditionals :cljs :clj now do the obvious expected thing in electric code blocks#2024-03-2113:09hkjelsI'm just getting to know electric, so I don't know what changes you are referring to I'm afraid.
Maybe it's actually really easy.. Something along the lines of this maybe.
(defn cljs-wrap
[{:keys [node]}]
{:node :reader-conditional
:forms [(assoc node :tag :cljs)]})#2024-03-2113:26simonkatzI’m doing this kind of thing:
(ns my-ns
{:clj-kondo/config
'{:linters {:unresolved-namespace {:exclude #?(:clj [cljs-only-ns-1
cljs-only-ns-2]
:cljs [clj-only-ns-1
clj-only-ns-2])}
:unresolved-symbol {:exclude #?(:clj [cljs-only-symbol-1
cljs-only-symbol-2]
:cljs [clj-only-symbol-1
clj-only-symbol-2])}}}}
(:require
...))#2024-03-2113:30simonkatzA bit clunky, perhaps, but at least it gets rid of the warnings.#2024-03-2114:17simonkatzAny thoughts on how to easily implement (wrap?) a nice color picker?#2024-03-2114:28xificurCWe have a fiddle if you want some inspiration
https://github.com/hyperfiddle/electric-fiddle/blob/main/src/electric_tutorial/demo_color.cljc#2024-03-2114:33simonkatzThanks. I will take a look later.#2024-03-2117:23simonkatzGreat — that helped.#2024-03-2120:51Daniel ManilaIs there a best practice for listening to a dom event in electric? I saw this https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_ui4.cljc#L105 but I'm not 100% clear on what it's doing (and I don't recall this being mentioned in the tutorial, but I could have missed it)#2024-03-2123:25Vincent@U06KR2E1LHM Which events were you thinking? You can listen to events on each of the ui components with nested (dom/on ...) for electric fns and (dom/on! ) for pure clojurescript
(dom/on "keydown" ...)
Sometimes I attach javascript to the dom to trigger functions on page load etc. For ui components the dom/on[!]s are good#2024-03-2405:43Andrew Wilcox[solved] Why does this code print "unmount" when the button is clicked?
#?(:cljs (defonce !state (atom 0)))
(e/def state (e/client (e/watch !state)))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(dom/div
(println "mount")
(e/on-unmount
(fn []
(println "unmount")
(prn state)
nil))
(dom/button
(dom/text "Click Me")
(dom/on! "click"
(fn [_]
(swap! !state inc))))))))
It doesn't appear to have actually been unmounted, as the div is still there. It doesn't appear to have been unmounted and then remounted, as "mount" isn't printed again.
The "unmount" message goes away if I remove the reference to the "state" variable. So maybe referencing reactive variables from inside of the unmount handler is bad? But then what would I do to get the current value of the state at the time of the unmount?#2024-03-2406:51HendrikAs far as I remember, on-unmount is more like a react use_effect #2024-03-2407:13noonianI think it's probably because when state changes, electric re-evaluates the e/on-unmount expression since it depends on state so the original on-unmount expression probably does get unmounted behind the scenes. I'm a decent way past the Balmer peak atm so please take all this with a large grain of salt 😅.
I suspect this is an interesting case that hasn't had a lot of thought from the electric team but I'm curious as to how one could determine if the callback was being called due to state changing or due to Main actually getting unmounted (assuming my explanation for why "unmount" is printed is correct).#2024-03-2415:07VincentI think noonian is correct#2024-03-2419:51henrikChange to prn @!state and it should go away.#2024-03-2423:17Andrew WilcoxOh, the fn has a reactive dependency on state. If I say (prn (fn [] state)) then prn is called each time state changes.#2024-03-2500:40Andrew WilcoxAs a followup to the previous, suppose I'm using
(e/for-by :id [o ...]
(Component. (:id o) (:data o)))
so that the id doesn't change during the lifecycle of the Component instance for that id.
(e/defn Component [id data]
(e/client
(dom/div
(println "mount" id)
(e/on-unmount (fn [] (println "unmount" id)))
(dom/text id ": " (pr-str data)))))
This works correctly because the id doesn't change.
Now suppose I mess up and forget to use e/for-by so that the id can change. I get confused. I'm seeing the "mount" and "unmount" messages, but the component isn't actually unmounting and remounting; the messages are getting printed again because of the reactive dependency on id.
Maybe I want something like
(e/defn Must-Not-Change [x]
(new (m/reductions
(fn
([] ::init)
([prev v]
(when-not (or (= prev ::init) (= v prev))
(throw (ex-info "value changed" {:previous prev, :next v})))
v))
(e/fn [] x))))
(e/defn Component [id data]
(e/client
(Must-Not-Change. id)
(dom/div
(println "mount" id)
(e/on-unmount (fn [] (println "unmount" id)))
(dom/text id ": " (pr-str data)))))
Now I get an error if my assumption that id doesn't change is incorrect.#2024-03-2502:59Andrew WilcoxOr better
(e/defn Component [id data]
(e/client
(let [!id (atom nil)]
(reset! !id id)
(dom/div
(println "mount with id" @!id)
(println "id is" id)
(e/on-unmount (fn [] (println "unmount with id" @!id)))
(dom/text id ": " (pr-str data))))))#2024-03-2503:01Andrew Wilcox(defmacro follow [e]
`(let [!v# (atom nil)]
(reset! !v# ~e)
!v#))
(e/defn Component [id data]
(e/client
(let [!id (follow id)]
(dom/div
(println "mount with id" @!id)
(println "id is" id)
(e/on-unmount (fn [] (println "unmount with id" @!id)))
(dom/text id ": " (pr-str data))))))#2024-03-2504:50Andrew WilcoxIt does seem like there are cases when I really don't want an expression to be reactive, and it'd be helpful to be able to make an expression nonreactive, or to be able to mark an expression as being supposed to be nonreactive so that I get an error if it actually is reactive.#2024-03-2505:15Andrew WilcoxAnyway, is the way I implemented follow here expected to be valid? Now and in the future? It appears to work for this simple example, but I don't what synchronicity guarantees Electric provides, if any.#2024-03-2507:29henrikIf you want something to be non-reactive, there’s e/snapshot#2024-03-2507:38Andrew Wilcoxe/snapshot sounds like a useful tool! It sounds like it would act differently than follow. The functionality I'm seeing with follow (if it's a valid implementation) is that on unmount it prints the current value of id, without calling e/on-unmount multiple times if id changes. Since the docstring for e/snapshot says it freezes the value, it sounds like it would also avoid calling e/on-unmount multiple times, but on unmount it would display the first value id had? I suppose I might want either in different situations.#2024-03-2507:43henrikYeah, sure. If you use snapshot, you get the value as of the moment that snapshot was initially called, and it won’t change for the lifetime of the closure it’s in. If you want to keep some value up-to-date that’s only relevant at the point that a closure unmounts, it’d be something like what you’re doing.#2024-03-2510:54Dustin Getzto make an expression non-reactive you can code it in a clojure function#2024-03-2510:55Dustin Getz((fn [] …)) also works - IIFE#2024-03-2510:56Andrew WilcoxYes, but how do I get a reactive value into the Clojure function without having the Clojure function called whenever the reactive value changes?#2024-03-2511:01Dustin Getze/snapshot (and i’ll have to try to understand your use case because we don’t like e/snapshot)#2024-03-2511:11henriksnapshot is useful for stuff like:
• You’re on “page A”.
• You send off a transaction that changes something on “page A”, and let’s say that it takes some time to complete this transaction.
• You switch to “page B” while the transaction is still happening
• Unless you snapshot the args to the transaction function, it will now re-run on “page B”#2024-03-2511:16Andrew WilcoxMy immediate question is whether this is a valid implementation of printing the current value of id at the time of unmount:
(e/defn Component [id data]
(e/client
(let [!id (atom nil)]
(reset! !id id)
(dom/div
(println "mount with id" @!id)
(println "id is" id)
(e/on-unmount (fn [] (println "unmount with id" @!id)))
(dom/text id ": " (pr-str data))))))
That is, this seems to work in my limited testing, but I don't know if the Electric execution model would mean that this is a correct implementation, or if it only works accidentally.
My more general question is about having the Electric component subscribe to an external event, and then call a Clojure function with the current value of a reactive variable when the event occurs. I notice for example that (dom/on! "click" (fn [] (foo id))) removes and re-adds the event listener when id changes. I'm curious if that's necessary and I'd need to follow the same pattern, or if I could subscribe on mount, unsubscribe on unmount, and pass whatever the current value of id is when the callback is called.#2024-03-2511:19henrikIf you go for a deref on the atom instead of the subscribed value, you’ll avoid recreating the handler function when ID changes. This can be quite valuable for values that are expected to change frequently. Imagine if you’re subscribing to some kind of clock or a stream of temperatures or something like that. You’ll end up creating and throwing away anonymous functions at a tremendous rate.#2024-03-2511:22henrikOr, if you’re using dom/on instead of dom/on! , this won’t be the case, since the reactive function isn’t recreated when the value changes. OTOH, you then need snapshot inside of the on -handler, or you’ll find that it re-runs if the value changes while the closure of the on -function is still executing.#2024-03-2511:23Andrew WilcoxDo you believe @U06B8J0AJ that my code sample in my previous reply is valid (it doesn't just happen to work accidentally)?#2024-03-2511:24henrikSo, to get exactly-once type semantics, with no recreation of handler functions:
(dom/on "click" (e/fn [_] … (e/snapshot id)))
or
(dom/on! "click" (fn [_] … @!id))#2024-03-2511:25Andrew WilcoxThe most important thing for me right now is to gain a conceptual understanding of how Electric works and what the execution guarantees are; not so much as to whether the particular implementation is the best one or not but whether it's correct.#2024-03-2511:27henrikSure. So in general, any reactive value will recreate the closure to which it is an input. If this “input” is into an anonymous function, it will recreate the anonymous function. If something is reacting to that function identity by executing it, it will re-execute. You can escape this, in general, with snapshot and deref . deref won’t re-run unless the identity of the atom changes (it follows the exact same rules as the rest on Electric).#2024-03-2511:30henrikTo be clear: there’s nothing magical about atoms, it’s just that they tend to be created once and then have a stable identity. Since deref only takes that stable identity, it won’t trigger re-runs. If you were to declare (let [!a (atom some-reactive-value)] , you’d get a new atom when some-reactive-value changes, and you’d find that your deref s re-run when that happens, since the identity of !a changed.#2024-03-2511:34henrikBtw, your code for printing out ID on unmount looks good to me.#2024-03-2511:36Andrew WilcoxAtoms I understand 🙂 I admit that I'm not seeing how my code could be incorrect. I suppose there might be some more complicated situation where the order of propagation of reactive values matters since I'm going outside of Electric's runtime, but I'm not thinking of an example right now.#2024-03-2511:37Andrew WilcoxDoes an e/fn have a lifecycle? If it does, what would cause it to unmount? If it doesn't, is it a (small) memory leak?#2024-03-2511:39henrikYes, it has a lifecycle, which is live for as long as its contents are live. So generally, the time it takes to execute the contents + the time of pending values inside of it.
For example, if you have a DB transaction that takes 2 seconds and is “pending” while that happens (e.g., with e/offload), the closure would be live for 2 seconds plus the execution time of whatever you do with the return value.#2024-03-2511:41henrikI’m describing that from the context of attaching it to a DOM handler. When invoked outside of a DOM handler, it remains live for as long as the closure in which it was called is live.#2024-03-2511:42henrike/defn ~= (e/def … (e/fn …))#2024-03-2511:43Andrew Wilcoxok, I'll need to play with e/fn 🙂 Thank you @U06B8J0AJ for your help!#2024-03-2512:04Dustin GetzTo be clear, the semantics Henrik describes here are semantics of dom/on, not e/fn itself, i.e. in e/offload), the closure would be live for 2 seconds plus the execution time of whatever you do with the return value.
#2024-03-2512:05Dustin Getzdom/on is doing low level stuff to simulate the idea of a callback with latency, which is a temporary solution until we provide a better pattern#2024-03-2512:05henrik> I’m describing that from the context of attaching it to a DOM handler.#2024-03-2512:08Dustin Getz(defmacro follow [e]
`(let [!v# (atom nil)]
(reset! !v# ~e)
!v#))
~the atom's nil initial value can be seen externally, this may be undefined behavior (wrt the nil)~#2024-03-2512:21Andrew WilcoxElectric expressions are Missionary continuous flows? And so always have a value? Thus the visibility of the atom's initial value isn't because the Electric expression might not have a value at first, but because the atom can be derefed before the reset! reactive expression executes for the first time?#2024-03-2512:23henrikYeah. If you can init the atom with (atom (e/snapshot id)) , it’ll never have nil , unless id is nil .#2024-03-2512:26Dustin Getzyeah the let has an implicit do which is concurrent, though now thinking it through fully, the do will be sampled which forces both child branches to be sampled, so for the ref !v# to escape, the reset! must have already occured#2024-03-2512:27henrikThat’s interesting. Will all branches in let be pending if any of them are pending @U09K620SG?#2024-03-2512:28Dustin Getz(do (throw (Pending.)) 1) is pending#2024-03-2512:29henrikRight, that was imprecisely phrased. The return value will be pending if any is pending.#2024-03-2512:34henrikSo, might print nil:
(let [!a (atom nil)
a (e/watch !a)]
(reset! !a 0)
(println a))
Never prints nil:
(let [!a (let [!a (atom nil)]
(reset! !a 0)
!a)
a (e/watch !a)]
(println a))#2024-03-2512:39Andrew WilcoxNever prints nil ?
(let [!a (atom nil)
a (e/watch !a)]
(let [_ (reset! !a 0)]
(println a)))#2024-03-2512:40Dustin GetzI want to dig in further into the first one:
• the do will initially boot it's children left-to-right (for Clojure compat), but reactive updates are concurrent / determined by the DAG
• that means, when the expression boots, (reset! !a 0) will run first and (println a) will run after
• but: will the watch update synchronously? I think yes it will, because clojure.core/reset! has clojure semantics and will notify all watch callbacks synchronously (from the clojure evaluation context) before returning back to electric
• that means by the time (println a) runs, the nil will be replaced, and never have been sampled by this expression#2024-03-2512:40henrikThat might print nil I believe (at @U05P1LZ8EQH), since reset! and println can race there, they’re independent of each other.#2024-03-2512:40Dustin GetzSo I guess it's not undefined here, though it is hard to generalize that without seeing a lot more examples and working through it#2024-03-2512:41Dustin Getz(I am rusty which is part of the problem, I haven't written code in 4 months)#2024-03-2512:47Dustin GetzI dont think _ and println can race in example 3, the electric `let` implementation touches each branch of the `let` for the sole purpose of making sure `(let [x_ (println 1) y_ (println 2)] nil)` matches Clojure's behavior#2024-03-2512:48Dustin Getz~I am not certain though now because I'm looking at a unit test that might say otherwise, i need to check with the team~#2024-03-2512:48Andrew WilcoxIn summary, follow is correct, but not for reasons I understand 😂#2024-03-2512:50henrikAha, interesting. I’ve been working with the assumption that from a closure and downwards into the tree, it’s graph rules: execution order depends ~on dependency order.
Even if it isn’t, I think I’d recommend that mindset anyhow, to avoid leaning too much on hard-to-explain subtleties.#2024-03-2512:51henrikI named the examples, a , b and c and executed them:
a nil
b 0
c nil
a 0
c 0
#2024-03-2512:52Dustin Getzhttps://github.com/hyperfiddle/electric/blob/45f7881df46f86e91a1730e893a05bed6cf4e728/test/hyperfiddle/electric_test.cljc#L333#2024-03-2512:56Dustin Getz@U06B8J0AJ i think you're referring to what i'd call the causal / pure functional / lazy evaluation order, which would imply that (let [_ (println 1)] nil) will not print as there is no consumer to sample it (other than the let special form itself), is that right?#2024-03-2512:59henrikNot exactly, more this:
[[:a :b]
[:b :c]
[:b :d]]
The execution order of [:b :c] and [:b :d] edges (and whatever subgraphs they might have) shouldn’t be thought of deterministic to each other, if in the same closure.#2024-03-2513:01henrikLike, if I feed that into a topsort algo, [:a :b] is going to be on top, but after that, it’s a coin toss.#2024-03-2513:05Dustin Getz(vector (vector a... b...) (vector b... c...) (vector b... d...))#2024-03-2513:06Dustin Getzargs to vector will be initially booted left-to-right, like do , and then are concurrent processes from there#2024-03-2513:07henrikRight, but even with the boot order, it’s a stronger claim to say that the CPU will get to the first vector before the second one, and that one will produce a return value before the other.#2024-03-2513:10henrikLike, I think pmap args launch in straight boot order, but trying to reason about what happens in the live context after that is perilous.#2024-03-2513:10Dustin Getzthe processes are concurrent once booted, though since missionary is evaluating them synchronously, there is no parallelism#2024-03-2513:11henrikReally, so they shouldn’t be thought of as async to one another?#2024-03-2513:11Dustin Getzso after the initial boot, if process d... updates, the memoized results from the other processes/objects are used#2024-03-2513:12Dustin Getz(div (div a b) (div b c) (div b d)) is more intuitive to think about#2024-03-2513:12Dustin Getzwe wanna minimize dom touches by maximizing concurrency#2024-03-2513:13Dustin Getzclojure.core/vector is foreign, so at the site of the foreign call it will need the memoized results to rerun the foreign fn (severing fine grained reactivity, the update is "clumped" now, collections bad)#2024-03-2709:18Hendrik(div (div a b) (div b c) (div b d))
What happens if b updates? Is there any ordering guarentee? And maybe more interessting: What happens here?
(div (println a-watch) (reset! !a-ref b) (println a-watch b)#2024-03-2709:27xificurCif these are electric semantic questions you can test them out through a couple of https://github.com/hyperfiddle/electric/blob/45f7881df46f86e91a1730e893a05bed6cf4e728/test/hyperfiddle/electric_test.cljc. E.g.
(tests
(with ((l/local (do (tap [a b]) (tap [b c]) (tap [b d]))) tap tap)
% := ...
% := ...
% := ...
))
You can add atoms and watches and swap!s to see changes. There's many tests you can look at or copy-paste and tailor to your question#2024-03-2509:39HendrikI have the following snippet:
(e/client
(let [s 40.0]
(e/server (println s (class s))))). ; prints "40 java.lang.Long"
I would expect, that this prints “… java.lang.Double” on the server
Is this is a bug or a limitation of clojurescript?
If I change 40.0 to 40.1, then “… java.lang.Double” is printed as expected#2024-03-2510:56Dustin Getzthis is javascript semantics#2024-03-2510:57Dustin Getzthere is no distinguishing between long and double in JS, 40.0 == 40, and 40 is the canonical representation#2024-03-2510:57Dustin Getzsame with JSON#2024-03-2510:58Dustin Getzthe transit serializer i believe is using JSON, so the relevant code is the transit java platform deserializer which is forced to make a choice and decides #js 40 on java platform should read as Long, and then your application has to parse it from there#2024-03-2518:10henrikIn plain Clojure, what’s the most direct way to take the first non-nil value from the result of a Missionary observe and joining it to the current thread?#2024-03-2518:11Dustin Getzsignals need an initial value so you're going to start Pending until the first non-nil value, is that your intent?#2024-03-2518:12Dustin GetzThere is always (when-some [x ...]#2024-03-2518:13Dustin GetzOr do you mean entirely without electric, pure missionary/clojure solution#2024-03-2518:14henrikEntirely without Electric, just bottoming out in an m/? , I surmise#2024-03-2518:15Dustin Getzthis is a #missionary question, I'd start with something like (m/eduction (comp (filter some?) (take 1)) (m/seed [nil 1 2]))#2024-03-2518:15henrikAh, sorry. I will repost there.#2024-03-2518:16Dustin Getzm/reduce to turn the flow into a task#2024-03-2518:17Dustin Getzhttps://github.com/dustingetz/missionary-quickstart/blob/main/src/quickstart.cljs has examples, just add an eduction#2024-03-2518:19Dustin Getzwhy do you have an m/observe if you just want the first value#2024-03-2518:19henrikAlright, I was on the right track then.#2024-03-2518:19henrik(m/? (m/reduce vector
(m/eduction (comp
(filter some?)
(take 1))
(m/observe (fn [!]
(let [p (foreign-proxy-async (keypath :attribute/string) pstate
{:callback-fn (fn [new _diff _old]
(println "New" new)
(! new))
:pkey :core})]
(fn []
(.close @p))))))))#2024-03-2518:20henrikI’m dealing with Rama asynchrony. The long and short of it is that I need a way to await some values being written before proceeding to write others, in a seed script.#2024-03-2518:21Dustin Getza more direct way could be to bind the callback to a m/dfv which is directly consumable as a task#2024-03-2518:21henrikI thought I’d do it with Missionary, because hey why not.#2024-03-2518:22henrikAh, great. I’ll try that.#2024-03-2518:26henrikdfv was totally the way to go, thanks ever so much#2024-03-2605:58Andrew WilcoxElectric without atoms: https://gist.github.com/awwx/6cf7971df7d856984a0b9fd47c804f40#2024-03-2608:55Andrew WilcoxAfter writing this, it occurred to me that a simpler approach than this sfn implementation might be to use a standard reactive fn, and then sample the fn value at the time an event occurs.#2024-03-2614:41Dustin Getzthis is clearly skilled work, can you tell me, what are you trying to achieve by avoiding atoms?#2024-03-2614:45Dustin Getzyour goal is an alternative way to subscribe to dom events with snapshot semantics built in?#2024-03-2614:46Dustin Getzor are you trying to achieve a specific unmounting-related problem that e/on-unmount doesn't help with?#2024-03-2700:55Andrew WilcoxMy goal at the moment is understanding the execution model 🙂 In the previous thread where I was asking if the way I was using atoms was correct, the answer was "yes, but due to very specific details of the execution model and if I make small changes it might break instead". So then I asked, well, do I actually need atoms? At all? Could I write code without atoms? If I did, would I find it easier to understand the guarantees?
There's the Java/JavaScript imperative model, the Clojure immutable data plus vars/refs/agents/atoms model, and the Electric reactive model. Whenever I write code to involves multiple models there's a greater chance that I'm going to get confused because I need to understand not only the models involved but also how they interact. If I have reactive state and I don't need to put it in an atom, if I have some reactive state that can happily live entirely within Electric, that seems like it could at least potentially be easier to debug: I just need to figure out what I'm doing wrong within Electric as opposed to Electric plus atoms plus the interaction between the two.
I don't think I'll end up using sfn as written, I think there may be other ways of doing that sort of thing if I end up wanting to for some reason... but I do find it interesting that son! only needs to attach the event listener at mount and remove it at unmount while dom/on! reattaches the listener when the fn updates. I don't know that it makes a practical difference, I'm not saying that there's anything wrong with reattaching the event listener on reactive updates... but, in terms of coupling between Electric and imperative JavaScript, being able to attach the event listener once seems like it could be a better fit... just, you know, probably not with sfn specifically, as that's pretty awkward.
Unmount was just an interesting edge case. I was generating and consuming an event stream, and one of the events I was generating was on unmount, but of course when I was processing events at the point of the unmount event the umount event wasn't getting processed because it was getting unmounted. So I was curious if I could rescue this by processing the event flow somewhere else that wouldn't be unmounted when the observed element was unmounted. Which I don't know if I'd ever need to want to do, but good to know that it's possible if I ever might want to for some reason 🙂#2024-03-2617:47HendrikHi, I have questions about server backpressure. I have this snippet:
(e/client
(let [!state (atom "")
state (e/watch !state)]
(ui/input "" (e/fn [v] (reset! !state v)))
(e/server (println state))
(println state)))
Now, I disable networking in chrome’s dev tools. Then I enter “asdfasdf” into the input element. Now I turn on networking again and only then
; a
; as
; asd
; asdf
; asdfa
; asdfas
; asdfasd
; asdfasdf
is printed in the server console.
What, if I only care about the latest value of state on the server after network is turned on again. How, can I achieve that?#2024-03-2701:47Vincent(ui/input (e/fn [] ...) ...) right?#2024-03-2701:48Vincentyou find backpressure undesirable is what you are saying#2024-03-2701:49Vincentyou want just the latest row in that output list is what you are saying#2024-03-2701:49Vincentui/input will emit on every on-change, you need to rely on a different element and a different event#2024-03-2707:27HendrikI want to discard values on network unavailability. This is what I found out from the docs:
In case of network unavailability the websocket buffer is filled and only then backpressure is propagated. The starter app sets the buffer to 100 MB, so it is unlikely to fill it. But even then you would only discard values, which are generated after the buffer was filled up.
Then I realized, that you cannot distinguish between slow network and network unavailability. So I came up with this solution so far:
(e/server (let [tx-flow (->> (e/fn [] tx)
(debounce 10)
(m/reductions {} nil)
new)] ...)
with the idea that pilled up messages come in fast, when network is up again.
In general I wanted to solve the problem, that I want to persist an input field with a cas transaction [:cas eid old-val new-val] and the initial value shown in the input field is the latest db value.
However, I realized that I have to deal with this client side and implemt some rollback behaviour#2024-03-2709:38Vincentcool. maybe important yea#2024-03-2712:28Dustin GetzI think this kind of thing should be done in the Electric network layer automatically. The server will see each message sent from the client (i.e. websockets are discrete), but if many changes to the same value arrive synchronously (i.e. network is "turned back on" and queues flush all at once) it does seem reasonable to flatten them, and it is semantically reasonable due to electric's continuous time semantics#2024-03-2712:28Dustin GetzIn other words, maybe we should log this as a bug#2024-03-2712:29Dustin GetzElectric v3 is coming which rebuilt the network layer and removed a ton of technical debt, maybe ask us again after v3 is delivered#2024-03-2703:49Andrew WilcoxHere's a version of dom/on! which provides a more idiomatic coupling on the JavaScript side: the event listener only needs to be attached on mount and removed on unmount. (Not that there's anything bad with reattaching the listener on reactive changes as far as I know, this is simply less surprising behavior). The strategy is to use (e/fn (fn [event] ...)) to produce a continuous flow of event handler functions, and then sample the latest one when an event occurs. https://gist.github.com/awwx/ec8e81c2682b0b67e7c0758dd100b3a9#2024-03-2707:07Geoffrey GaillardThis is great. I've been giving this one a go too, for performance reasons.
An alternative approach is to store successive callback values in an atom. On event fired, deref the atom and call the callback. No need for a continuous flow.#2024-03-2707:30HendrikI have another question about pending values:
(e/client
(let [server-value (e/server (case (e/offload #(m/? (m/sleep 1000)))
true))]
(try
(new (->> (e/fn [] server-value)
(m/eduction (comp
(unique-transducer)
(map #(js/console.log %))))))
(catch :default e
(println "is pending")))
(println "print" server-value)))
This prints:
;immediatly
hyperfiddle.electric.Failure {error: h…e.e…c.Pending}
; after 1 second
true
cprint true
Why is the pending value injected into the “server-value” flow, but not in the reactive (println “print” server-value)?#2024-03-2708:23Geoffrey GaillardAll electric expressions but try/catch will short-circut on hyperfiddle.electric.Failure instances, like Pending. Missionary, which is lower level, will not.
In your example you are intercepting the Pending value, and your eduction retruns nil.
try doesn’t receive (doesn’t see) any Pending, it only sees nil.#2024-03-2708:45HendrikAh I see. I changed the code, so that it is clear, what is going on
(println "print" (new (->> (e/fn [] (do (println "lambda" server-value) server-value))
(m/reductions #(do (print "log" %2)
%2) nil))))
; prints
log #object[Object [object Object]] ; pending object
log #object[Object [object Object]] ; pending object
lambda true
log true
print true
So the (println “lambda” server-value) and (println “print …) are only executed, if none of the args is of instance e/Failure?
What about e/fn? This does not short circuit, right?
And about no missionary short circuit: Does the compiler look at the vars namespace and takes special care of all missionary/xxx things?#2024-03-2709:26Geoffrey GaillardI’m not sure about e/fn.
> So the (println “lambda” server-value) and (println “print …) are only executed, if none of the args is of instance e/Failure?
Yes, akin to the Either monad, fmap will short-circuit on Left (Failure) values.
The compiler doesn’t treat missionary code any differently. Electric error-handling happens at runtime.#2024-03-2709:32xificurCa good mental model - electric doesn't have exceptions. try/catch is sugar over an either monad. Interop calls (clj/cljs) cannot run if any parameters are a Left (Failure) value. Electric code can operate with Failures and due to absolute concurrency runs everything it can run. E.g. (do (throw (Exception.)) (println :reached)) will print, but the result of the do is a Failure#2024-03-2709:37Hendrikthank you for your replies.
And for the missionary part:
(m/reductions rf flow) is called like a normal function and the “flow value” (e/fn [] value) is created once and does not change if “value” changes, right?#2024-03-2709:39Hendrikso the flow is handled inside the reductions function and therefore outside of electric context - like anything in a normal clojure function 🙂#2024-03-2709:41Hendriknow I got it 🙂 thank you so much#2024-03-2709:43Hendrikand for (e/fn [] value). Probably it will short circuit inside, too, and emit a pending failure, which is not short circuited because it is handled inside missionary :thinking_face:#2024-03-2709:46Geoffrey Gaillarde/fn lifts an electric expression into a missionary-compatible flow.
Its dual is new, which joins flows into electric.
In your example, new is the first short-circuting point#2024-03-2709:48xificurC(m/reductions rf (e/fn [] value)) will see Failures, since the value is thunked. We use it in e.g. the implementation of https://github.com/hyperfiddle/electric/blob/45f7881df46f86e91a1730e893a05bed6cf4e728/src/hyperfiddle/electric.cljc#L545 (`r/pending` is (Failure. (Pending.)))#2024-03-2709:53Hendrikyes. My question is more about what happens inside e/fn if value is r/pending. Does emit value or does it shortcircuit and emit a r/pending? This is more out of curiousity because an outside observer, who consumes this flow, can’t tell the difference, right?#2024-03-2709:57xificurCcan you give an example of what you mean?#2024-03-2710:04HendrikI try:
(let [value r/pending
>flow (e/fn []
(println value) ;inside an electric function, so short circuit expression on failure
value ; normally the electric function would return /emit value, but this is r/pending; does it return value or just emit an r/pending?
)]
(new (m/reductions #(println %2) >flow))
))
I think, that this is an implementation detail because a flow consumer would not notice the difference#2024-03-2710:07xificurC(let [value r/pending]), in Either Right/Left speak value is Right[Left[pending]], probably not what you were after. (throw (Pending.)) is how you create a Left[pending] in electric#2024-03-2710:08xificurCif value = r/pending I don't see the difference between returning value or r/pending#2024-03-2710:13HendrikThanks. Now I have a much clearer picture. And you helped me to keep my sanity. I was thinking on these questions for a whole day. 🙂#2024-03-2713:07Absolute NegativityHow to handle e/offload interruption?
(let [load (e/server
(try (let [uid (d/entid db [:user/name "SOME-PEEP"])]
(e/offload
(fn [] (expensive-call db uid))))
(catch Pending e "PENDING")))]
(dom/div (dom/text (pr-str load))))
In this setup, when I fired two db transactions I would get an interruption error and load would never recover.#2024-03-2713:09henrik(try (expensive-call db uid)
(catch InterruptedException _ nil))
#2024-03-2713:11Absolute NegativityOh I see. That simple. Thank you!#2024-03-2713:13henrikIt might be worth wrapping your DB transactions with interruption handling somewhere further in so that you don’t have to remember to do this every time you want to transact something.#2024-03-2713:24Dustin Getzi wonder, should e/offload swallow that exception?#2024-03-2713:28henrikIt’s a fairly common one. It fires like crazy when sending searches to Lucene, for example. If it’s swallowed such that it has no further impact on the Electric program, that might be useful.
The way we handle it currently is by making it the only case that returns nil (as opposed to [] , which is “search completed, but no results”). We can then drop those frames. Not dropping on nil causes search results to flicker back and forth while typing.#2024-03-2713:33henrikBasically, this is the dance we do:
(defn search
[…]
(try (search-lucene …)
(catch InterruptedException _ nil)))
(e/server
(let [res (search …)
[search-results set-search-results!] (use-state [])]
(when res (set-search-results! res))
(e/client
;; Do something with "search-results"
…)))#2024-03-2713:44Dustin Getzare you sure the flickering is caused by duplicate nils and not Pending ?#2024-03-2713:54henrikWithout the try/catch, it would interrupt Electric completely and end up in a broken state. Catching the exception fixes that problem, but introduces intermittent nils. Pushing them through an atom fixes that problem.#2024-03-2714:12Dustin Getzyou claim that repeated nils cause flickering?#2024-03-2714:15henrikYeah. It would be […] nil […] nil nil […] etc.#2024-03-2714:16Dustin Getzoh!#2024-03-2714:17henrikIt so happens that the search results cache serves another purpose btw: it’s placed far up enough that everything except it unmounts when search is closed, which makes the prior search available immediately when reopening search.#2024-03-2714:17Dustin Getzmove the atom into the search fn so you can write (catch InterruptedException _ cached)#2024-03-2714:17Dustin Getzthe nil is your fault#2024-03-2714:19henrikWhat purpose would the cache serve inside the search function? It would be recreated as empty each time search is called.#2024-03-2714:20henrikOr do you mean, pipe in the contents on the cache to the search function? That would work.#2024-03-2714:21Dustin Getzyeah or a macro#2024-03-2714:23henrikThat would admittedly be slightly cleaner, but like I said, we consider it a feature to be able to place the cache further up the tree from the search, to remember the previous search when search is reopened.#2024-03-2714:23Dustin Getzok#2024-03-2714:24henrikNo, thinking about it, it doesn’t invalidate your point. I could still give the cache to the search function to use as a fallback 👍#2024-03-2714:25Dustin GetzFor the record is the slow piece needing a cache Lucene or Electric#2024-03-2714:27henrikIt’s not about slowness, it’s whether you want the search to open to a clean slate, or open such that it contains the previous search.#2024-03-2714:28henrikBoth valid options, but we opted for remembering the previous search.#2024-03-2714:29henrikIf we wanted a clean slate and didn’t have to catch InterruptedException, the cache would be unnecessary.#2024-03-2717:17weicould use some help debugging what i think is a SLF4J deps issue. i get this runtime exception starting electric+rama from an uberjar:
% java -cp app.jar clojure.main -m prod
SLF4J: Class path contains multiple SLF4J providers.
SLF4J: Found provider [org.apache.logging.slf4j.SLF4JServiceProvider@57eab3f5]
SLF4J: Found provider [ch.qos.logback.classic.spi.LogbackServiceProvider@698dd40c]
SLF4J: See for an explanation.
SLF4J: Actual provider is of type [org.apache.logging.slf4j.SLF4JServiceProvider@57eab3f5]
Unexpected problem occured during version sanity check
Reported exception:
java.lang.AbstractMethodError: Receiver class org.apache.logging.slf4j.SLF4JServiceProvider does not define or inherit an implementation of the resolved method 'abstract java.lang.String getRequestedApiVersion()' of interface org.slf4j.spi.SLF4JServiceProvider.
at org.slf4j.LoggerFactory.versionSanityCheck(LoggerFactory.java:331)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:172)
...
at clojure.lang.RestFn.invoke(RestFn.java:1099)
at electric_starter_app.server_jetty$eval8517$loading__6803__auto____8518.invoke(server_jetty.clj:1)
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
...
here's the repro, just added rama deps to electric starter repo: https://github.com/yayitswei/rama-electric-issue-repro
this is the deps tree: https://gist.github.com/yayitswei/391c511927bfb0de4537d7b470423893
what's a good way to manage overlapping deps here?#2024-03-2722:28Vincentyou did not upgrade your mac os right#2024-03-2722:29Vincentoh i see you are thinking overlapping deps between rama and electriC?
hmm :thinking_face:#2024-03-2800:20weiupdate: i realized it's more of a warning since the server still starts. but might affect logging#2024-03-2814:47Dustin Getzi get this warning in all configurations including dev: SLF4J: Class path contains multiple SLF4J providers.#2024-03-2814:51Dustin Getzit kinda looks to me like rama is on an old version of slf4j (1.7, 1.8) and electric is on 2.0#2024-03-2814:52Dustin Getzfrom the deps tree you showed, electric gets slf4j via ring/ring 1.11.0 (which is new in jan 2024, we upgraded around this time)#2024-03-2814:53Dustin Getzspecifically ring/ring-jetty-adapter 1.11.0#2024-03-2814:57Dustin Getzok, in dev i resolved SLF4J: Class path contains multiple SLF4J providers. by allowing rama to provide the logger, excluding it from ring, and not listing logback-classic explicitly, which resolves to slf4j 1.8#2024-03-2815:11Dustin GetzOk, in uberjar, at runtime this still results in ERROR StatusLogger Unrecognized format specifier [d]#2024-03-2815:11Dustin GetzI tried <root level="OFF"></root> in the electric logback.xml but still error#2024-03-2815:12Dustin Getzit appears to be a logger format string but not from the electric logback.xml#2024-03-2815:12Dustin Getzi grepped the unzipped uberjar, no logback.xml, so i'm not sure where the format string is coming from#2024-03-2815:13Dustin Getzrama being closed source (and also fat) makes this hard to debug, have you checked with rama team? I dont think this is an electric issue#2024-03-2815:20Dustin GetzI dont see the error when launching like clj -M:prod -m prod#2024-03-2815:29Dustin GetzOk, i was successfully able to produce (on purpose):
➜ wei-rama-electric-deps-issue git:(main) ✗ java -cp target/app.jar clojure.main -m prod
Unexpected error (ClassNotFoundException) macroexpanding proxy at (ring/adapter/jetty.clj:137:3).
org.slf4j.LoggerFactory
#2024-03-2815:29Dustin Getzafter a clean build:
clj -X:build:prod uberjar :build/jar-name "target/app.jar"
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
shadow-cljs - server version: 2.26.2 running at
shadow-cljs - nREPL server started on port 49496
[:prod] Compiling ...
{:optimize true, :debug false, :verbose false, :hyperfiddle.electric/user-version "fc0a886-dirty"}
[:prod] Build completed. (154 files, 0 compiled, 0 warnings, 11.72s)#2024-03-2815:29Dustin Getz#2024-03-2815:44Dustin Getzoh shit i got it maybe
java -cp target/app.jar clojure.main -m prod
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
{:optimize true, :debug false, :verbose false, :hyperfiddle.electric/user-version "fc0a886-dirty"}#2024-03-2815:44Dustin Getzi excluded org.apache.logging.log4j/log4j-slf4j18-impl from rama#2024-03-2815:45Dustin Getz#2024-03-2815:45Dustin GetzI also cloned ring and tried to track down the logging strategy to see if there is some kind of v1 vs v2 version conflict causing the crash within ring, but i couldn't find much#2024-03-2815:46Dustin Getzanyway it is now clear from the original error, that org.apache.logging.slf4j.SLF4JServiceProvider is the source of the "version mismatch" issue, it looks like a facade#2024-03-2815:47Dustin Getzorg.apache.logging.log4j/log4j-slf4j18-impl -> appears married to v 18#2024-03-2815:48Dustin Getzbtw this is all with electric's logback.xml set to
{:tag :configuration, :attrs nil, :content ("\n " {:tag :root, :attrs {:level "OFF"}, :content nil} "\n")}#2024-03-2815:51Dustin Getzin summary, if ring-jetty-adapter 11 is ultimately incompatible with rama, it is possible to downgrade and make electric work on the older ring jetty adapter again, but it's not something we can afford to support unless it is profitable for us, so you'd be on your own until Rama upgrades, and due to the weight of all Rama's transitive java dependencies that seems unlikely to happen. So that would be bad news#2024-03-2815:53Dustin Getzon the other hand, everything seems to work fine with clj -M:prod -m prod as your launcher, do you really need an uberjar or is it force of habit?#2024-03-2816:03Vincentwhat does rama bring to the table that electric doesn't already do#2024-03-2904:24weiit's a complementary backend for electric's reactive frontend#2024-03-2904:41weiyes, unfortunately rama's pretty heavy-handed with the forced dependency versions. fortunately everything still runs despite that error. also, worst case i can separate out rama and electric into two projects.#2024-03-2904:42weiso anyways no action needed, and really appreciate all the investigative work!#2024-03-2904:44weido you really need an uberjar or is it force of habit?guess not. the uberjar is just for demonstration purposes since it's using rama's IPC. in production i believe the rama deps should be marked as provided and not bundled in the jar#2024-04-2018:10siddharth yadavThis worked for me
https://clojurians.slack.com/archives/C05N2M7R6DB/p1712865271327839?thread_ts=1707521559.365899&cid=C05N2M7R6DB#2024-04-2608:40weithanks for the tip! that worked for me for getting rid of the java.lang.AbstractMethodError at startup.
still seeing these sorts of errors in the log though:
ERROR StatusLogger Unrecognized format specifier [d]
please lmk if you have any additional findings!#2024-03-2801:55Andrew Wilcoxlifecycle on-mount/on-unmount that works with reactive values: https://gist.github.com/awwx/fa44158661520ea68fc0742b3c905589#2024-03-2806:58HendrikWhat is the difference between lifecycle and e/on-unmount?#2024-03-2810:10Andrew WilcoxThe fn passed to e/on-unmount has a reactive dependency on any reactive values referenced within the fn. So for example
(e/on-unmount
(fn [] (println "e/on-unmount child 2 (broken reactive)" k)))
the "e/on-unmount" message gets printed whenever k changes, even though the node hasn't actually been unmounted.#2024-03-2811:31Hendrikah and your lifecycle will only run on unmount, but when it runs it uses the latest value of the unmount function, right?
I see the usecase for on-unmount, but what about mount in Lifecycle. Isn’t that redundant to electric functionality?#2024-03-2811:38Andrew WilcoxYou could use e/snapshot, but with lifecycle you don't have to.#2024-03-2804:18Andrew WilcoxI think I may have confused someone here... this is ok:
(defn foo []
(let [mbx (m/mbx)
f (m/ap (m/amb nil (m/? mbx)))]
nil))
but
(e/defn Foo []
(let [mbx (m/mbx)
f (m/ap (m/amb nil (m/? mbx)))]
nil))
doesn't compile:
Syntax error macroexpanding cloroutine.core/cr.
Could not resolve var: mbx
{:var mbx, :file "NO_SOURCE_PATH", :end-column 36, :column 33, :line 181, :end-line 181}
ExceptionInfo: Could not resolve var: mbx
(It's not a problem for me because I can use foo)#2024-03-2805:05Andrew WilcoxNot that I have a reason to embed the Missionary DSL in the Electric DSL, so probably better to not anyway.#2024-03-2813:31Dustin GetzLooks like a bug to us#2024-03-2813:31Dustin Getzwe have not seen this before#2024-03-2813:32Dustin Getzmissionary is mostly electric-compatible, it is only the m/ap m/cp macros i believe that are incompatible due to the cloroutine macroexpansion using mutable arrays (which we haven't looked into yet)#2024-03-2808:38HendrikI have created a missionary delayed flow. It does -> delay -> emit -> delay -> emit ….
(defn delayed [delay >input]
(m/ap
(m/? (m/sleep delay))
(let [x (m/?> >input)]
(m/amb
x
(let [_ (m/? (m/sleep delay))]
(m/amb))))))
this works if you use it in raw missionary, but fails if used in electric like this:
(->> (e/fn [] [reactive-value])
(m/reductions {} nil)
(m/relieve
{})
(delayed 4000)
(m/reductions #(println %2) nil)
new)
it runs, if I remove the first (m/sleep), but I want the initial delay before being ready to transfer
(defn delayed [delay >input]
(m/ap
#_(m/? (m/sleep delay))
(let [x (m/?> >input)]
(m/amb
x
(let [_ (m/? (m/sleep delay))]
(m/amb))))))
I have no idea why it fails (stacktrace in the comment)#2024-03-2808:40Hendrik{:message "Unable to build frame - not an object.", :data {:hyperfiddle.electric.debug/trace [{: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["[]" <exception>], :hyperfiddle.electric.debug/origin #uuid "ddd00842-2375-490e-a7b7-af6163ffbe6a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["0" <exception>], :hyperfiddle.electric.debug/origin #uuid "ddd00842-2375-490e-a7b7-af6163ffbe6a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["#function[missionary.core/reductions/fn--14215]" <exception>], :hyperfiddle.electric.debug/origin #uuid "ddd00842-2375-490e-a7b7-af6163ffbe6a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [<exception> <exception>], :hyperfiddle.electric.debug/origin #uuid "ddd00842-2375-490e-a7b7-af6163ffbe6a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/origin #uuid "ddd00842-2375-490e-a7b7-af6163ffbe6a", :hyperfiddle.electric.debug/args [], :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["true" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:valid true, :value true, :error-msg nil, :initial-value true, ...}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:valid true, :value true, :error-msg nil, :initial-value true, ...}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:valid true, :value true, :error-msg nil, :initial-value true, ...}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:value true, :tx-id 1021}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["#object[cljs.core.Atom {:val #}]" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["true" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:value true, :tx-id 1021}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:value true, :tx-id 1021}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:value true, :tx-id 1021}" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["true" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {},
#2024-03-2808:40Hendrik:hyperfiddle.electric.debug/args ["true" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["1021" <exception>], :hyperfiddle.electric.debug/origin #uuid "d862371e-a622-4afa-bf93-1316a4f4e87a", :hyperfiddle.electric.debug/serializable true} {: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args ["{:account/primary-email \"#2024-03-2810:17Andrew WilcoxI don't know if this is the problem, but flows lifted into Electric with new need to have an initial value. You could make the value pending for the first delay.#2024-03-2810:20Andrew WilcoxMaybe something like (m/ap (m/amb r/pending (do (m/? (sleep delay)) ...
(r is hyperfiddle.electric.impl.runtime)#2024-03-2810:22HendrikI tried that. I get the same error#2024-03-2810:31xificurChard to say without a full repro at this point. As a last attempt you can try the simplified version below#2024-03-2810:50Hendrikthe simplified version works. If I add a (m/? (m/sleep ms)) before the first fork, then I get an error. I suspect, that it has to do with the fact, that the flow does not accept values from >in immediately. It would have been great to know the cause, but I was able to work around this by using a modified version of e/debounce.
and again: thanks for your help. I really appreciate that 🙂#2024-03-2814:10simonkatzI have a user input. When the value changes, a bunch of dependent stuff runs. One of those things is a call to a DB query on the server. When that returns, the rest of the UI is updated.
Suppose I know or suspect that values in the DB have changed and that the query, if re-run, will return new results.
Is there a way to cause the dependencies of the input to re-execute even if the input hasn’t changed?#2024-03-2814:15Dustin Getzan incrementing time parameter , ideally you can derive it from the database in some way#2024-03-2814:20simonkatzSo nothing I can do from within a dom/input / dom/on to mark the input as dirty when the user presses the return key?
Not sure about the time thing. I think I’d want the update to happen because of a user action.#2024-03-2814:43Dustin Getzi dont understand#2024-03-2814:43Dustin Getzif you want to rerun the query (function) the only way to do that is to pass a changed parameter to it#2024-03-2815:02simonkatzOK, that answers my question; thanks.
I think maybe I can add an extra piece of state, a timestamp for when the user presses the return key, and pass that as a parameter in the server call.
I’ll give that a whirl.#2024-03-2815:04simonkatzSo my watched atom will contain both the user input itself and the timestamp.#2024-03-2816:04Vincentlast-touched might be a good name for that timestamp#2024-03-2901:45Panelhttps://twitter.com/dustingetz/status/1773035945379484041#2024-04-0213:17JIs there a repo about that?#2024-03-2917:29weithis input is supposed to add a new item, then clear itself on blur.
(ns test.test
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-ui4 :as ui]
[hyperfiddle.electric-dom2 :as dom :refer [div text props on]]))
#?(:clj (def !items (atom ["a" "b" "c"])))
(e/defn Test []
(e/client
(div (text (pr-str (e/server (e/watch !items)))))
(let [!v (atom "")
*v (e/watch !v)]
;; new item ui
(ui/input
*v
(e/fn [s] (reset! !v s))
(on "blur"
(when-not (empty? *v)
(e/fn [e]
(e/server (swap! !items conj *v)) ;; add new item
(reset! !v "") ;; clear the input for the next new item
)))))))
why do i get this error?
#error {:message called `new` on nil, :data {}}
when I comment out the (reset! v! "") on the last line, i don't get the error anymore.#2024-03-2917:58noonianIf *v is blank, you are registering nil as the blur handler. I think you probably want to check for blank inside your blur handler efn#2024-03-2918:03weigood catch, thanks! the e/fn should wrap the when-not not the other way around.#2024-03-2920:55Adrian SmithHas any one got flow storm debugger working with electric?#2024-03-2921:49Adrian Smithoh I might be hitting this
https://clojurians.slack.com/archives/C03KZ3XT0CF/p1711736387725389#2024-03-3015:31Absolute NegativityCan I make e/server ignores unserializable value and pass it to the outer e/client?
(e/client
(uix/$ antd/Dropdown
{:menu {:items
(e/server
(e/for-by :entity/id
[entitiy (e/server (find-entities state/db))]
(e/client
{:label (uix/$ :div (e/server (:entity/id entity)))
:key id})))}}))
Here e/server would complain about the React component in item's labels. I'd like to have the full db entity in server context when constructing the React component.#2024-03-3105:10Vincentis the nested e/server intentional#2024-04-0307:25henrikYou need move the for-by to the client and fetch each entity individually within the for-by. Alternatively, build up a complete payload server-side, then loop over it on the client.
In Electric, the mounting of DOM elements is a side effect produced by calling the dom/xyz constructors, which is why this would work if it were pure Electric code. In the case of UIx, it wants the return value of the for-by , and produces no side-effects. Hence the return value of the for-by becomes important, and how it passes between client and server matters.#2024-04-0303:02Vincentso in theory if one is gonna use electric to have a chat bot / ai friend, chat with / dialogue with you, how to make the results "stream" from the api endpoint?#2024-04-0305:39weii asked a similar question 6 months ago and it resulted in this discussion https://clojurians.slack.com/archives/C7Q9GSHFV/p1695368808283179#2024-04-0305:40weifor the API side, you can use or reference this lib that supports streaming https://github.com/wkok/openai-clojure#2024-04-0305:53VincentVery wonderful of you to share so readily the successful result ty @U066TMAKS#2024-04-0306:26henrikWe were advised to accumulate into an atom. Basically, an m/observe which concats the result into a server-side atom, which is what we ended up doing.
The downside is that since you stream the accumulated result of "This", "This is", "This is an", "This is an AI", "This is an AI respon", "This is an AI response" and so on as the response is built up, the accumulated payloads ends up being the size of some power of the final length of the AI response. From what I measured, a ~1.5kb response ends up costing ~270kb on the wire. It works OK UX-wise, but keep an eye on your outgoing bandwidth costs if your thing gains traction.#2024-04-0314:59joshcho2we accumulated into an atom as well#2024-04-0315:00joshcho2used openai-clojure#2024-04-0315:52weihere's a full implementation for reference: https://github.com/groundedsage/clj-chatbot-ui (ht @U05095F2K). it accumulates into an atom as well#2024-04-0323:45Andrew WilcoxSuppose on the server we had a reactive value chunks containing a vector which gets extended as new values come in: ["This", " is ", " an", " AI", " respon", ...]. If we tried (e/server (e/for [chunk chunks] (e/client (dom/text chunk)))), I wonder what that would look like on the wire?#2024-04-0506:55VincentGentlepeople who fiddle for a living, this article about a violin maker is really inspiring (found on HN)
https://archive.ph/ymimZ#2024-04-0311:03Andrew WilcoxHere's an invalid program:
(defn not-a-continuous-flow [<f]
(m/ap
(m/amb
"prefix"
(m/?> <f))))
(e/defn Main [ring-request]
(e/client
(let [!a (atom 1)
a (e/watch !a)
<a (e/fn [] a)
>b (not-a-continuous-flow <a)
b (new >b)]
(prn b))))
#object[Error Error: Unable to build frame - not an object.]
Easily fixed of course with m/relieve. I'm curious, how does Electric even notice that this isn't a continuous flow? It seems like some discrete flows such as m/seed can be fed to Electric. Is the broken assumption here that the output flow won't be producing more values than the input flow?#2024-04-0504:02Thomas CothranI'm taking a look at electric, and I have a few beginner questions.
1. How easy is it to add electric in to an existing Clojure ring server, keeping the existing routes intact. Looks like it should be doable from the examples.
2. How easy is it to try electric out in an existing reagent application? For example, to render a single component on a page, where the rest of the application uses reagent (or any other fe framework)
3. What's the current status with respect to form handling? Are there any major limitations to be aware of?
4. Since there is in memory server state, isn't it problematic when a server node goes down?#2024-04-0505:18Andrew WilcoxRegarding #4, the demos show server state being simply stored in memory, but naturally for a real system you would have your server connect to some persistent data source such as a reactive database. Then like any system where a client is making reactive subscriptions to a server, the client can reconnect and resubscribe.
A different issue is that in Electric the client DOM is built based on the subscription to the server. There's no diffing between a desired state of the DOM and the actual state of the DOM like React does. So right now, at least in dev mode, when the client reconnects it rebuilds the DOM from scratch. The DOM itself has state, such as text entered into a text input, which element has the focus, etc., and that state is lost when the DOM is rebuilt. I don't know if there's a plan for this... I suppose the client could keep track of DOM state and reapply it on reconnection, as one possible approach.#2024-04-0506:26henrikWe did the journey from UIx to Electric. We still keep some UIx components around.
1. This part was pretty straightforward. Look at https://github.com/hyperfiddle/electric-starter-app and follow that.
2. Since you can only (easily) have a single Electric instance for an app, I would suggest the opposite, which is what we did: put Electric at the top of the application, then mount your current app as a (full-screen) component inside Electric. From there, you can e.g. convert the app shell/frame to Electric, while keeping the inner Reagent components, and in that way work your way down into the app.
3. Do you mean network communication wise, or UI wise? Neither should be particularly problematic as far as I know.
4. To add on to Andrew’s answer, one of the great things about Electric is that it very aggressively tears down state built up in its closures. This means that it’s far more tenable to do things like holding a medium sized blob of data in-memory server-side, while showing a window into it client-side. This type of thing would be fraught with danger outside of Electric, since you have to be mindful to clear the server-side state yourself, or you will have a memory leak. With Electric, client-side state or server-side state have the same semantics and guarantees, so it’s just a design question. And like Andrew points out: if you want something to be persistent, you’ll have to persist it.
If the server goes down, or the client connection is lost, the app will remount and re-render. The extent to which the app remounts to the same state as before the connection is lost depends on how much of the app state is captured in something stable or persistent, like the current URL, indexeddb or the backend database.
Offline mode and server-side rendering is currently out of scope for Electric.#2024-04-0517:51Thomas CothranThanks for the pointers!
The reason I asked a third question, about form handling, was I saw a slack message saying that form handling was a weak point and was being worked on. And I did not know if that was still the case, or what exactly the limitations are/were.
Putting electric at the top is interesting, I hadn't really considered it. My initial thinking here was to try electric out in an unobtrusive place before doing anything like committing to a rewrite.#2024-04-0600:54VincentI think working from Electric-as-a-basis and selectively importing routes from your other application would be a good strategy. Moving electric into an application is hard.#2024-04-0600:54Vincentmoving non-electric things into the electric app is significantly simpler to reason about.#2024-04-0600:54Vincentand has the bright side of connecting one with the latest electric version#2024-04-0601:04Thomas CothranHmm, is it fair to say then the electric is not a good fit for a micro-frontend setup, where the back end would be a microservice in the front end would be inside a web component?
Or even a microservices set up without the micro frontends, since you would have multiple back ends serving data to a front end?#2024-04-0613:10Vincentthere's no such thing as front-end and back-end. There is the Electric program. The electric program insists you colorize/tint aspects of code as (e/client or (e/server
these are loosely synonymous with frontend/backend but differ in many key ways:
There is no network layer to access.
I mean, there is, if you want to do CLJS-AJAX from the e/client blocks for example, it is possible and I do it in some projects, but if you consider the "how do i pass it between client and server" the answer is : you don't have to! you make an electric variable that stays synchronized in the scope to either client or server over the socket, invisibly to you as the developer#2024-04-0613:13VincentThe approach is heavy handed in order to ensure it can enable many beautiful things:
+ network collapse / total database on client transparency
+ synchronized updates of values on client/server making the client essentially have a "global database" glued onto it on the side
+ multiplayer out of the box#2024-04-0613:15Vincentdepending on what you want to make, you should consider how much realtime talk over the wire is desirable, and you should consider if cranking the firehose of network connectvitiy all the way to FULL is the correct choice for whatever application you have in mind when you ask the question. Electric is brilliant and works for 3 of my projects so far, but there has been a lot of massaging to make it work as a standalone product. I am wondering what you are making, microservice is vague, if you can provide more info I'd be delighted to provide more guidance.
If you cannot or don't want to: or would like to know what I would say regardless of your goals in particular at the moment:
Tonsky's Rum is great for writing components in a reusable way with react-keys and it's possible to do differential over-the-wire updates with something like Datomic ... but it's a lot more book-keeping for you / the coder#2024-04-0623:54Thomas CothranI'll give an example: suppose you have a large case management system that has several teams of developers working on it. These teams maintain their own microservices, which each have their own repository.
One micro service might be responsible for billing, another for time entry, another for contract management, etc.
The trouble is that the UI talks to multiple services. That is to say there's not a one-to-one . That is to say there's not any one service the browser would be making calls to. And there's not one service responsible for the ui.
And ideally each team is deploying their front end changes separately of the others, although in practice things can be tied together by a front end application. But still, there's not one team dedicated to the front end, all the teams are making the front end changes for the features they own.
I'm trying to see how electric could fit in that situation. None of the teams are dedicated to just the UI.
To give an example by contrast, one of the things I've experimented a bit with is a live view style approach, where there's a web component whose state is controlled on the server, and as that state on the server changes diffs are sent down a websocket, and the front end updates. In this case, it's relatively easy to drop in web component and control it from a microservice. And you can have multiple on a page.
Hopefully that example doesn't obscure more than illuminates#2024-04-0806:47henrikNo dedicated FE resources sounds like it might create UX challenges in the app, but that’s a separate question. In practice, I don’t see why separate teams couldn’t own separate parts of an Electric app just as it does today.
Electric is agnostic to how you manage statefulness, so each team can still be responsible for updating their part of the app. It doesn’t care how many databases or services you talk to in order to render the app. However, Electric is a network optimizer. In order for it to optimize communications, you have to actually use it for this purpose. If all your app does is to POST to different endpoints directly from the client, you’re going to be losing out on a lot of the benefits.
> To give an example by contrast, one of the things I’ve experimented a bit with is a live view style approach, where there’s a web component whose state is controlled on the server, and as that state on the server changes diffs are sent down a websocket, and the front end updates. In this case, it’s relatively easy to drop in web component and control it from a microservice. And you can have multiple on a page.
Electric does exactly this: it’ll manage reactivity, diff, minimize payloads, JIT disparate pieces of server-side processing so that they converge at the right moment on the client etc. It has to be aware of the whole program at compile time to do this though, so the Electric app has to be centrally located and fully redeployed when it’s changed. You can’t serve an Electric app piecemeal from X different servers. However, if the microservice comms were to be moved to the server side of the Electric “service”, then it could manage both UI reactivity and network payloads for you.#2024-04-0819:38Vincent@U06B8J0AJ in your response did i read the hint of a "maybe we can electric without knowing the whole app in advance" ?#2024-04-0906:11henrikAha, that was unintentional in that case. No, it needs to see the whole app in advance as far as I’m aware.#2024-04-1001:55VincentOkay just checking haha 😄#2024-04-0514:59grounded_sageExperiencing a weird bug where I use ui-comp in a couple of (e/server ..) calls and it then fails at using it a certain amount of times. (e/for-by identity [ui-comp (sort component-list)] ...)#2024-04-0515:00grounded_sageIt almost seems like it just loses the reference to it.#2024-04-0515:02grounded_sageThis works
(swap! !server-layouts update-in (concat layout-selector [:config :order])
(fn [state]
(println "(remove #{ui-comp} state): " ui-comp #_(vec (remove #{ui-comp} state)))
state)
This fails
(swap! !server-layouts update-in (concat layout-selector [:config :order])
(fn [state]
(println "(remove #{ui-comp} state): " ui-comp #_(vec (remove #{ui-comp} state)))
(println "the ui comp:" ui-comp)
state)#2024-04-0515:06grounded_sageWorking around it by moving this out into a seperate clj function call. But does seem like a bug in electric.#2024-04-0813:40Geoffrey GaillardWe've been working on a datafy/nav based Electric datagrid. Premise: if one can datafy it, Electric can render it.
Features:
• efficient virtual scroll (server streamed rows, only in the client-visible scroll window)
• datafy/nav rendering as forms and tables, with input filter and column sort - see screenshot
• malli schema based rendering
• progressive enhancement
Demos:
• https://electric-datagrid.fly.dev/(datagrid.gitbrowser%2FGitBrowser) – https://github.com/hyperfiddle/electric-fiddle/blob/ed6e907111fd12d4d3f7966649c09b545dcfdcd4/src/datagrid/gitbrowser.cljc#L77
• https://electric-datagrid.fly.dev/(datagrid.file-explorer%2FFileExplorer) – https://github.com/hyperfiddle/electric-fiddle/blob/ed6e907111fd12d4d3f7966649c09b545dcfdcd4/src/datagrid/file_explorer.cljc#L73
• https://electric-datagrid.fly.dev/(datagrid.datomic-browser%2FDatomicBrowser) – https://github.com/hyperfiddle/electric-fiddle/blob/ed6e907111fd12d4d3f7966649c09b545dcfdcd4/src/datagrid/datomic_browser.cljc#L291
Request for feedback: We're going to spend a bit more time working on these datagrid apps, what should we build?#2024-04-0813:40Geoffrey GaillardIdeas:
• Datomic schema explorer
• Clojure namespace browser
• JDBC explorer
• Hackernews comment browser
• S3 browser
#2024-04-0813:49Dustin GetzWe're seeing lag from here in US East that we don't experience in Paris, we're investigating, unclear if fly issue or what#2024-04-0813:57henrikThis is what it looks like from Stockholm (using the worst broadband provider in Sweden)#2024-04-0813:59henrikPing @U02DVTY209E, check it out#2024-04-0814:00Dustin GetzWe currently believe it's the disk IO latency on the fly instance due to clj-git doing file system ops#2024-04-0814:00henrikCheat and load it into memory#2024-04-0814:02Dustin GetzAdditionally, making the window half screen (25 rows not 60) is much faster across all demos#2024-04-0814:03wikipunkThis is really cool thanks for sharing #2024-04-0814:07henrikLimiting number of rows made a huge difference. What do you reckon is the bottleneck @U09K620SG? Something DE might improve, or completely orthogonal: network latency, disk perf?#2024-04-0814:09Dustin Getzsuspect combination of network-attached cloud disk latency on the git demo (which is by far the worst), + overhead of electric v2 doing too many round trips (v3 is better and has the right infrastructure in place now to start working on algorithmically optimal network for which the design work is already done), and also the overhead of datafy/nav doing blocking IO outside of e/offload which we need to look into#2024-04-0817:04Ben SlessIdeas - generic database explorer with configurable sinks - not just s3 and jdbc, but redis, aerospike, druid, clickhouse, graphql#2024-04-0817:06Ben SlessAnother idea - hijack some code analysis tool and make a code base explorer (maybe metazoa or clerk?)#2024-04-0817:12henrikIn that vein, tap> target?#2024-04-0819:41VincentSuper cool!!#2024-04-0822:48nivekuilnot the feedback you asked for, but I can't see myself using any of these purely on a UX perspective (keyboard controls, column x row filters etc.) -- maybe you could think about a common interaction system for datagrids#2024-04-0822:53nivekuilput another way, less focus on getting developers excited to make stuff with your system and more about how to get the end users to demand developers make stuff with your system#2024-04-0822:53nivekuilnot sure if anyone currently likes being on the receiving end of low code tools#2024-04-0907:59Geoffrey GaillardAll feedback is welcome, especially the one we don’t expect. You are definitely right. We are prioritizing what we consider the hardest, most difficult or most risky to outsource/offload. I’m sure we’ll prioritize differently soon enough.
> not sure if anyone currently likes being on the receiving end of low code tools
Food for thoughts 👌#2024-04-0909:53PanelData table, dropdown, forms ect.. are full of very hard UX / accessibility issue, in the react world a proper data grid implementation has probably more dev working on it than the whole hyper fiddle team.
#2024-05-2906:59grounded_sageCan one play with any of this yet? @U2DART3HA
I'm currently doing a lot of work with data that I am rendering into tables and feel like I am just recreating a subset of what this is.#2024-05-2907:09Geoffrey GaillardSure, have a look at the latest electric git browser: https://github.com/hyperfiddle/electric-fiddle/blob/demo%2Fdatagrid/src%2Fdatagrid%2Fgitbrowser.cljc#2024-05-3000:20Dustin Getz@U05095F2K fair warning - this stuff isn’t mature and may be difficult to use, and there can be performance issues in nontrivial configurations#2024-05-3002:58grounded_sageAppreciate the warning. #2024-06-0718:32JAtkinsBTW some of your git sha dependencies seem to be breaking. Took a minute to find out that the SHA from that branch of eletric-fiddle depended on a missing ref in electric. Eventually discovered it's using electric -> feature/experiments#2024-04-0814:31Daniel ManilaIs there a way to call an e/fn directly? I admit that I don't fully understand how they work. I'm trying to do something like the following, where I only call a handler if it exists:
(e/defn Inner [handler]
(e/client
(dom/div (dom/on! "click" #(when handler (handler)))
(dom/text "Click me"))))
(e/defn Outer []
(e/client (let [h (e/fn [e] (println "Does this work?"))]
(Inner. h))))#2024-04-0814:32WillYou cant call the electric function inside of a clojure function. If you change the dom/on! to dom/on and use an e/fn instead of #() it should work#2024-04-0814:33WillOther than that, I think it should work?
That being said, in your example, h is a pure clojure function, so you could define it without using e/fn and use it directly with dom/on! in Inner#2024-04-0814:34Daniel ManilaHmm, I tried this code and got the error n is undefined
(e/defn Inner [handler]
(e/client
(dom/div (dom/on "click" (e/fn [e] (when handler (handler))))
(dom/text "Click me"))))
(e/defn Outer []
(e/client (let [h (e/fn [e] (println "Does this work?"))]
(Inner. h))))#2024-04-0814:34Wille.g.
(when handler (dom/on! "click" handler ...#2024-04-0814:34Daniel ManilaCan you pass pure clojure functions to electric components, I think that wasn't working for me earlier#2024-04-0814:35Will(e/defn Inner [handler]
(e/client
(dom/div
(when handler
(dom/on! "click" handler))
(dom/text "Click me"))))
(e/defn Outer []
(e/client (let [h #(println "Does this work?")]
(Inner. h))))#2024-04-0814:35Daniel ManilaOh wait you can, I guess I was having another issue#2024-04-0814:35Willtry something like that, should hopefully work#2024-04-0814:37Daniel ManilaYeah seems to work. Thought I had tried that, but thank you for talking me through!#2024-04-0814:39WillIn your last code snippet you passed in an electric function but called it like a clojure function (handler). If you wanted it to be an electric function, you could still make it work like this:
(e/defn Inner [handler]
(e/client
(dom/div
(when handler
(dom/on "click" handler))
(dom/text "Click me"))))
(e/defn Outer []
(e/client (let [h (e/fn [e] (println "Does this work?"))]
(Inner. h))))#2024-04-0814:39Willnote the change from on! to on#2024-04-0814:42Daniel ManilaAny tips on how to debug errors like the following:
ERROR hyperfiddle.electric: #error {
:cause fexpr__49599_49651 is null
:data {:hyperfiddle.electric.debug/trace [....]}
:via ...}
I'm getting it in code that appears to be working, so I'm not sure what's failing#2024-04-0814:46Geoffrey GaillardDoes the error remains after a REPL restart?#2024-04-0814:48Daniel ManilaIt does unfortunately#2024-04-0814:48Geoffrey GaillardIs this happening in dev mode or under advanced compilation?#2024-04-0814:48Daniel ManilaDev mode#2024-04-0814:49Daniel ManilaI cloned the electric-starter app, so I've been using its (dev/-main) to start up the compiler/server#2024-04-0814:52Geoffrey GaillardI’m thinking about a client / server program mismatch, which usually do not happen.
It could be due to:
• a cljc namespace failing to compile in clojure but succeeding on cljs.
• a macro macroexpanding differently in clj vs cljs.
Does it happen immediately after the repl is started? or after saving a change in a file?#2024-04-0814:53Daniel ManilaIt happens when I'm calling a function (roughly the same pattern as https://clojurians.slack.com/archives/C7Q9GSHFV/p1712586660579119, but something seems to be different since it's not working properly)#2024-04-0814:59Geoffrey GaillardCan you show me the code?#2024-04-0814:59Daniel ManilaYeah let me try and figure out how to display it, I haven't put it up on github yet#2024-04-0815:15Daniel ManilaHere is the line that's getting https://github.com/DonyorM/Tree-King/blob/main/src/great_dalmuti/components/hand_wheel.cljc#L40occurs (I think, the error isn't super clear)
The stack trace references the https://github.com/DonyorM/Tree-King/blob/main/src/great_dalmuti/components/card.cljc, since it's calling that handler#2024-04-0815:41Geoffrey GaillardYou are calling Card with a clojure.core/fn :on-click, but inside Card you are calling (dom/on "click" on-click) and (dom/on "tap" on-click).
dom/on takes an e/fn as argument
dom/on! takes a regular fn as argument
Try to change (dom/on "click" on-click) with (dom/on! "click on-click) in Card.cljc#2024-04-0815:43Daniel ManilaThat did it, sorry that I seem to keep making that mistake#2024-04-0815:43Geoffrey GaillardThe error is not clear at all and is not helping, for sure.
We will improve it.#2024-04-0815:44Daniel ManilaThanks for making this library, it's certainly been fun to learn with#2024-04-0815:44Geoffrey GaillardTurns out today it is not that obvious for the compiler to detect fn vs e/fn and report an error. Hence it’s hard to produce a good error message.#2024-04-0815:47Daniel ManilaYeah that makes sense. I worked on a project to improve clojure error messages a while back and our main conclusion was that it was harder than we'd thought 😛#2024-04-0905:16grounded_sageThe latest electric-starter-app doesn't seem to reload styles for me. I used to put it directly into index.html which worked fine on my previous repositories. This also doesn't trigger an automatic update of my styles (dom/link (dom/props {:rel :stylesheet :href "/styles.css"}))
I am wondering if I am missing something that changed to get the css to automatically update.#2024-04-0905:35Geoffrey GaillardI don't think anything changed in that regard. Maybe look into the shadow-cljs :watch-dirs config entry?#2024-04-0905:41grounded_sageActually this may be an issue with my computer. It's definitely on it's way out.#2024-04-1016:08weiI ran into this too. maybe check your config (https://github.com/hyperfiddle/electric-starter-app/blob/main/src-dev/dev.cljc#L17)? the starter app defaults to
:resources-path "public/electric_starter_app"
which is probably not where you're putting your styles.css. i'm not a fan of the electric_starter_app dir refactor because I have to change it in multiple places every time I clone the starter repo to create a new app.
also, in case you haven't tried this, I added this to my :dev profile in shadow-cljs.edn
:devtools {:loader-mode :default
:watch-dir "resources/public/app"}
#2024-04-1020:47Dustin Getzwe are looking into repairing this#2024-04-1108:35Geoffrey GaillardFixed on starter app’s https://github.com/hyperfiddle/electric-starter-app/commit/18bef64 by adding :watch-dir do shadow-cljs.edn.#2024-04-1009:27teodorluAre there anyone here using Electric - but not across network boundaries, just on the JVM or just on Javascript? If yes, consider sharing why in 🧵! (asking out of pure curiosity, I have not used Electric on just the server or just the client)
Please put yes yes or no ⛔ on this post!#2024-04-1105:26noonianI was using Electric exclusively on the client to evaluate it as a replacement for React in frontend apps when a websocket connection is undesirable#2024-04-1201:05denikhi! before the zero config entry point it used to be possible to bind dynamic vars during a server request like so
(defn wrap-electric [handler]
(fn [req]
(if (:websocket? req)
(let [user (req->user req)]
(http-kit/as-channel
req
(electric/handle-electric-ws
req
(fn [write-msg read-msg]
(binding [db/*user-ent* user]
(electric/electric-ws-message-handler req write-msg read-msg))))))
(handler req))))
I tried this now but it’s not working:
(defn wrap-electric-websocket [next-handler]
(electric/wrap-electric-websocket
next-handler
(fn boot [_req]
(binding [db/*user-ent* (req->user _req)]
(e/boot-server {} tesserae.ui.app/Main)))))
e/boot-server takes opts. can some of these be var bindings? would be great to find a workaround since part of my app is broken without it.#2024-04-1213:51Dustin GetzI think the question is: how to bind Clojure dynamic scope (e.g. user) in the entrypoint that specifically is not Electric dynamic scope? (Because you could pass the request into Main and bind the user from inside the Electric program but this doesn't convey into your Clojure functions, and for some reason you can't or don't want to pass it into them as an argument)#2024-04-1213:51Dustin GetzIs that correct?#2024-04-1213:52denikyes#2024-04-1213:52denikand re: binding inside electric, I can try that#2024-04-1222:53denikthis worked! thanks!#2024-04-1213:47Dustin GetzGit client POC – four hyperfiddle datagrids over a server side git repo + css layout + very light customization#2024-04-1213:48s-olI'm seeing some unexpected (inconsistent?) behaviour with conditionals, here's a small repro:
(e/server
(let [!txs (atom [1 2 3])
txs (e/watch !txs)]
(e/client
(when (not-empty txs)
(ui/button (e/fn [] (e/server (swap! !txs drop-last)))
(dom/text "toggle" txs))
(e/server (prn "txs" txs))))))
every time you click the button the list gets shorter. when it becomes empty the button disappears.
But it does log "txs" () , in other words: code inside the when ... block is executed even though the condition is falsy
is this expected? I'm not sure how to guard against this, since it can caus exceptions in the downstream code (where the print is)#2024-04-1214:03s-olbonus problem was that something ate the exception caused by downstream code (where the prn is) and only printed generic electric error traces#2024-04-1214:04s-olonce I understood that this state-shearing was the problem I could move the logic that crashed into a larger clojure-func with it's own when guard as a workaround, but I don't think I should have to do that?#2024-04-1214:06s-olfor reference, here's that exception trace. Can't provide a minimal repro for it though.
user=> ERROR hyperfiddle.electric-ring-adapter: Websocket error
java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
at hyperfiddle.electric_ring_adapter$electric_ws_handler$on_message__25754.invoke(electric_ring_adapter.clj:160)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$apply.invoke(core.clj:662)
at hyperfiddle.electric_ring_adapter$ring_ws_handler$fn__25762$fn__25763.doInvoke(electric_ring_adapter.clj:181)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.websocket$eval25501$fn__25505.invoke(websocket.clj:13)
at ring.websocket.protocols$eval25266$fn__25267$G__25251__25276.invoke(protocols.clj:3)
at ring.adapter.jetty$websocket_listener$reify__26113.onWebSocketText(jetty.clj:76)
at org.eclipse.jetty.websocket.core.internal.messages.StringMessageSink.accept(StringMessageSink.java:53)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.acceptMessage(JettyWebSocketFrameHandler.java:348)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onTextFrame(JettyWebSocketFrameHandler.java:436)
at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onFrame(JettyWebSocketFrameHandler.java:241)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor.lambda$onFrame$1(WebSocketCoreSession.java:671)
at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1466)
at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1485)
at org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker$1.handle(AbstractHandshaker.java:212)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession$IncomingAdaptor.onFrame(WebSocketCoreSession.java:671)
at org.eclipse.jetty.websocket.core.AbstractExtension.nextIncomingFrame(AbstractExtension.java:145)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:236)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.emitFrame(DemandingFlusher.java:145)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher.inflate(PerMessageDeflateExtension.java:487)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension$IncomingFlusher.handle(PerMessageDeflateExtension.java:413)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.process(DemandingFlusher.java:169)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:243)
at org.eclipse.jetty.util.IteratingCallback.succeeded(IteratingCallback.java:369)
at org.eclipse.jetty.websocket.core.internal.DemandingFlusher.onFrame(DemandingFlusher.java:104)
at org.eclipse.jetty.websocket.core.internal.PerMessageDeflateExtension.onFrame(PerMessageDeflateExtension.java:93)
at org.eclipse.jetty.websocket.core.internal.ExtensionStack.onFrame(ExtensionStack.java:120)
at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession.onFrame(WebSocketCoreSession.java:481)
at org.eclipse.jetty.websocket.core.internal.WebSocketConnection.onFrame(WebSocketConnection.java:271)
at org.eclipse.jetty.websocket.core.internal.WebSocketConnection.fillAndParse(WebSocketConnection.java:464)
at org.eclipse.jetty.websocket.core.internal.WebSocketConnection.onFillable(WebSocketConnection.java:349)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
at java.base/java.lang.Thread.run(Thread.java:840)
#2024-04-1215:02xificurCwe know of a bug we dubbed "when true", the underlying cause is a deeper design issue. It is an edge case, i.e. it happens rarely. It is fixed in the new v3 design. In v2 we had a workaround but I can't find a quick recipe for you. Can you try to change the conditional from (when (not-empty txs) ..) to (when-some [txs (not-empty txs)] ..)?#2024-04-1215:03xificurCto be clear I'm not 100% sure your issue hits this bug, but its shape is similar#2024-04-1215:47s-olthe repro in OP has the same result when rebinding txs using when-some or when-let#2024-04-1215:48s-olmy workaround in my actual project is reasonably tasteful (putting the calculation into its own function was a decent move for readability anyway) and I've only hit this in that single case so far so I'm ok waiting for v3 I guess#2024-04-1221:14Vincenta few things that come to mind:
when should receive one argument aka (do ...)
(e/client ... typically the very next thing I see is (dom/div ...) so maybe e/client straight to conditional and multi-line when is rascally.
You could potentially go something like:
(e/server
(let [!txs (atom [1 2 3])
txs (e/watch !txs)] ;; this is defined within an (e/server
(e/client
(when (not-empty txs) ;;could put when first, then (dom/div
(dom/div (dom/props {})
(ui/button
(e/fn [] (e/server (swap! !txs drop-last)))
(dom/text "toggle" txs))
(e/server (prn "txs" txs)))))) ;;when do you want this to run? #2024-04-1221:19Vincentthat last line with (e/server ..) looks like an inscrutable not-simple-to-reason-about "when will this happen" -- maybe you can stash this into the (e/fn [] statement) on the (ui/button ..)#2024-04-1508:12s-olI want that (e/server ...) blob to run every time txs has a new value, but only if that value passed the (when (not-empty txs)) check#2024-04-1508:14s-olwhen and do explictly exist to do multiple things, no? Otherwise you can just use if (arity 2) and nothing at all. In general one of the strong points of electric is "composes just like regular clojure" so I don't think there's anything wrong in principle with this code#2024-04-1508:16s-olIf I were to make a guess, it seems like v2 has consistent updates for value changes, but due to the lazy and async nature of electric, mount/unmount are not (always) consistent with values. I can see how that is not a straightforward thing to achieve and looking forward to v3#2024-04-2715:33grounded_sageI am getting a similar issue
ERROR hyperfiddle.electric-ring-adapter: Websocket error
java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
There is a lot happening in the project I am on so actually scoping this issue isn't really possible. Would need more information from the exception to find where the error is occuring.#2024-04-2715:50grounded_sageIs there anyway to get a deeper trace to get some information of where the error comes from?#2024-04-1220:55bhurlowI have have an existing data source that live updates (say a websocket connection or evented filesystem), what’s the best way to pass that data into electric? Is it best practice to just e/def an atom wrapped in e/watch?#2024-04-1223:50Dustin Getzyou would likely use missionary for this and then join the missionary flow to electric with new #2024-04-1223:51Dustin Getzm/observe is a good place to start, check out my missionary quickstart #2024-04-1502:53bhurlowperfect thanks, that’s kinda what I assumed#2024-04-1610:29simonkatzI think I have a problem with ui/scroll-state. See details in my in-thread follow-up.#2024-04-1610:29simonkatzI have a gridsheet based on https://github.com/hyperfiddle/electric/blob/master/src/contrib/gridsheet.cljc#2024-04-1610:32simonkatzIt’s generally working well.
I noticed that the e/for at https://github.com/hyperfiddle/electric/blob/45f7881df46f86e91a1730e893a05bed6cf4e728/src/contrib/gridsheet.cljc#L67 supplies key values in the range 0 to (dec page-size), rather than providing values that depend on the rows that are being displayed.
I tried to fix this using a for-by with a key that is always the same for the same data row.
But that causes my UI to continuously redraw, each time changing which is the top line.
That’s happening because the value of scroll-top (returned by ui/scroll-state) keeps changing.
Any ideas on what the cause of this is?
More detail…
The video at https://drive.google.com/file/d/1rZwVG0TqnGPx17feSNmCtdIHR-sq5zTJ/view shows what happens.
A cut-down version of my code is at https://github.com/simon-katz/nomis-electric-for-by-problem/blob/f750b83f59e66b82816d0c6a60696840ae0fcf7c/src/nomis_electric_demo/main.cljc#L105
That code could probably be cut down further — quite possibly I don’t need all the grid stuff to demonstrate the issue. If I don’t get any joy in response to this, I’ll try cutting that down.#2024-04-1615:00Dustin Getzoff the cuff answer - electric v2 has a problem, which is why we work around it by stabilizing the row containers with fixed keys. The problem IIUC is that unmounting dom nodes, horrifyingly, pays round trip latency, due to a design flaw in v2. It should be fixed in v3, we're just about mature enough to start testing this use case#2024-04-1615:10simonkatzAh, OK; thanks.
I’ll move on to other things. 🙂#2024-04-1615:10Dustin GetzDid you have a reason for trying to change the strategy?#2024-04-1615:12simonkatzI thought it was the correct thing to do and that it might make things faster.#2024-04-1615:12simonkatzBut it works well enough#2024-04-1615:12simonkatzSo not a problem#2024-04-1615:12Dustin Getzok#2024-04-1717:30Vincentelectric clojure + ai agent .. anybody craft a PoC yet?#2024-04-1804:07mattiasWe're using electric in building uxo.ai. How do you envision electric being used in this case?
We're using it to build the UIs for controlling and interacting with uxo, but a really powerful use will be to pull all the extracted data to a knowledge base whose interface is built with electric (the knowledge base would be similar to some ideas I outlined here https://github.com/subspace-ai/subspace). So you can build simple ad hoc interfaces on top of your knowledge base from simple electric functions, and the knowledge base can easily ingest data from the Web and desktop apps via uxo.#2024-04-1804:08mattiasWe're also open to collaborators, particularly people who have used electric and are into AI 😊 #2024-04-1818:06Vincent> How do you envision electric being used in this case?
Sidecar AI agent with you at all times connected over socket. And, soon, multimodal language models means you can talk and it will talk back.#2024-04-1818:11Vincentcomposable differentiable UI elements=components will be choice.
Tracking global state in an elegant way will be key.#2024-04-1818:11Vincentlet me take a look at your github and see what i can offer if anything 😅#2024-04-1818:16mattiasthe github repo doesn't have much, but the readme outlines an idea for a PKM+REPL+RAG app, into which I could see http://uxo.ai ingesting data from the web and other desktop apps. which is the kind of interface i'd like to have for an AI agent#2024-04-1818:17VincentYeah what is a sh0rtcut is it a jupyter notebook style integration that fetches data and runs code a'new? is it a buncha mini baby lektrik kompilers that live inside the mothership Compiler?#2024-04-1818:17Vincent(or what are some potential ways you envision it)#2024-04-1818:27Vincent@U05095F2K out of curiosity have you made this yet 😄#2024-04-1818:27mattiasdidn't quite understand what you mean, but let me explain a bit more
i envision sth like subspace-ai (as described in the readme) which is your roam-like PKM. inside the PKM, you can have (1) regular text nodes (like in roam), (2) code nodes, which you can execute manually like in a REPL or a Jupyter cell (where the output could be an e/fn that gets rendered), and (3) code nodes that execute an e/fn and continuously render some UI element, possibly based on the data that is in the PKM or any other data source. I would use this PKM+REPL in and of itself like i use roam + some things i currently run from the terminal, e.g. triggering some ML training job etc, and would build custom dashboards with info that is relevant to me, nothing more
now, in addition to that there is http://uxo.ai, which is a separate product (that https://www.linkedin.com/in/mattias-arro-b63a9110/ and my https://www.zafstojano.com/ are working on). Currently we are not much focused on integrating with or building out subspace-ai, but i could see a useful integration where from subspace-ai I call into uxo-ai to either fetch data from the web, or interact with some web or desktop UI and parse/visualise the results from subspace-ai
we are also looking for people who know electric to join uxo, so if this general topic seems interesting we could have a chat#2024-04-1818:31VincentRoam is super cool and, new info for me. Let me have a closer looksie. And also, sounds cool! I need to step away and think about all this for a sec.#2024-04-1818:35mattiasof course 🙂 there are interesting alterntives to Roam Research, particularly tana.inc, which also has some ai features but is more focused on the less technical user who doesn't need as much control as us programmers would want#2024-04-1818:47VincentThe idea behind uxo where I could scrape and also create new interfaces for the data is mind blowing.#2024-04-1818:47VincentTruly composable data sources as interfaces#2024-04-1818:48Vincentto quote one of my favorite songs, that shi cray.#2024-04-1818:49VincentI was thinking about how I can pre-load an AI agent, connect them over socket, talk to them like I would talk to a normal user, but then have my user also able to talk with them.
I mainly want ai agent over socket, but integrating a mini ai agent as an operative function would be clutch.#2024-04-1819:24Vincentfor example, with the uxo use-case
• GPT3.5 or haiku scrapes the data,
• assembles some sort of reusable data form[at]
• creates some electric components with source and css revealed in browser frames
• tell the component to be "live" or "static / wait for clicks"
#2024-04-1821:30VincentI suppose hitting the api is just-as-good-as having a live agent to query, at this moment in time. What's the process of scraping or subscribing to data? (question i have) and also what is the ideal output (RSS for example, was a new standard that was adopted joyfully by many). Would it be better to issue a new electric internet standard for live querying differential updates#2024-04-2108:32mattias@U055PQH9R4M Currently you can use uxo locally on macos. It comes with an Electr*on* app that just renders an Electr*ic* UI that is a Jupyter-like interface to run simple web automation code snippets. But it also exposes an nrepl port. The code snippets control a Chromium browser and can scrape, autonomously navigate and perform basic actions on the web. The key is these automation scripts have an AI layer built in, which allows you to define what data structures get extracted (using a Pydantic-like schema library) and then return these extracted data structures or use them to further drive your automation script. The AI layer makes the automations generic enough that they can work across a broad range of websites, usually. You would use this desktop app to develop / debug some automations you need to run, or run personal automations which e.g. require your login and need a human-in-the-loop workflow.
For example, one such automation might go to a url, navigate to the page that has articles, go through each article and fetch+returns titles and full text of articles as json. Such automations can be auto-generated based on plain English prompt of what's needed. Below is Python, but Clojure will also be available probably since the core is written in clj.
class NewsStoryShort(BaseModel):
title: Optional[TextNode] = Field("The title of the news story")
text_short: Optional[TextNode] = Field("A short introduction of the news story")
time: Optional[TextNode] = Field("The time of the news story")
full_link: Optional[List[LinkNode]] = Field("The link leading to the full news story")
download_link: Optional[List[LinkNode]] = Field("The download link of the news story")
class NewsStoryFull(BaseModel):
title: Optional[TextNode] = Field("The title of the news story")
text_full: Optional[TextNode] = Field("Full text of the news story")
time: Optional[TextNode] = Field("The time of the news story")
class ArticlesPage(BaseModel):
news_story_short: Optional[List[NewsStoryShort]] = Field(
"A container element for a news story article."
)
def scrape_articles(src_url):
uxo.page.navigate(src_url) # set url on location bar
uxo.ai.navigate_to( # use AI-driven navigation
"page with a list of articles",
example_hrefs=[
"/articles",
"/stories",
"/investor-relations",
"/about",
],
max_num_pages=5,
)
ret = []
for ret in uxo.ai.page_extract(ArticlesPage):
for article in ret.short_articles:
article.full_link.click()
article = uxo.ai.page_extract(NewsStoryFull)
ret.append(article.to_dict())
uxo.page.back()
return ret
In terms of api-based access, there is lots to do on our end, but eventually you can deploy the above fn as an autoscaling API endpoint and POST it with different URLs (or set up input/output queues to do that). Alternatively the POST request could contain the automation code itself, for ad hoc use cases.
As you can see UXO is not your regular agent, but a much more controlled way to define web automations that's a step less rigid than traditional scraping tools. Interacting with it is still procedural, a request-response cycle (where the response is a JSON in the structure defined by the user). However if you integrate that into a PKM type of tool (like subspace-ai), you could keep the (RAG / in-context kind of) chat-with-your data inside your PKM, which is IMO where it belongs, and use UXO to just ingest data to the PKM.
You could, however, use the primitives built into UXO to build sth like an RSS feed over one or many websites. For example for one customer we're building scrapers that can gather data from various event providers, where the scraped Event and EventType entities will be PUT to XTDB. Here Event is sth like a "concert" and EventType is "concert", "theater performance" etc. And the frontend, which is currently still ad hoc for the customer, is simply displaying the resulting entities in a UI with Electric.
So in our case, we subscribe to data via XTDB and build a custom UI on top, since there will be additional controls in terms of how to interact with the scraped events. But, I have thought that maybe sth like a customisable HFQL or similar approaches as in the datagrid demos could be use here to make these ad hoc UIs easy to define while allowing full-blown Electric based UI components which are completely custom. I'm not sure about "new electric internet standard for live querying differential updates", would love to hear your thoughts on it.#2024-04-2108:34mattias@U09D96P9B I saw their docs mention "agents" but the demo video is just about interacting with different LLM APIs. Do you know if they have sth ready?#2024-04-2402:59VincentCool.
I have a question: is there a way or a roadmap way to restrict agent access to data to very very specific parts?
I'm thinking about how to incorporate AI agents generally into an application that has lots of not-shareable customer data and it would be catastrophic to have a front-end-facing AI-agent introspect the data and reveal it.
so, how to : have front-end facing AI agent and also, not share all the data#2024-04-2406:09mattiasCurrently there is no such feature, but this seems like a sensible thing to add, shouldn't be too difficult. There could be two types of restrictions:
• URL regex, which the agent would never navigate to (or if it did due to some JS redirect, it would terminate)
• configurable CSS selectors, which contain sensitive info; the agent would mask these selectors out in the image/html input to the model. if you control the html of the app, just add class="sensitive" and we can filter it out
Such heuristic restrictions are the only ones that can provide real guarantees, but in this case it seems a reasonable approach.
Perhaps we should set up a call to discuss your use case?#2024-04-1804:08JAtkinsAnyone tried to do a recursive viewer on data? Portal-esq. I've tried a couple times but I get stuck in how abstract the code is and get hung up. Optimally, it'd be similar to portal where users can override the viewer at a certain level#2024-04-1911:54braai engineerMinimal Multiplayer Mouse Cursors in Clojure ala Tonsky's blog: https://github.com/theronic/electric-multiplayer-mouse/blob/main/src/electric_starter_app/main.cljc
Live here on a 1GB VM: http://41.79.79.230:8080/
Try open multiple tabs to see more cursors if no one is online.
(VM bandwidth tiny so might fall over)
Mouse cursor position stream not optimal – I'm sure it can be debounced for better perf.
Cursors rendered as SVG.#2024-04-2000:37Dustin Getzit’s gonna be slow from most locations due to round tripping the mouse to SA, you’ll need to manually optimize for local cursors#2024-04-2116:29VincentBravo#2024-04-2208:04HendrikTried it. Can confirm what Dustin says. ~ 0,5 s lag from Central Europe. I would handle the local cursor in a client atom. If you you want to be super fancy, you can do a local prediction of the server cursors’ positions from the last known positions, too 🙂#2024-04-2412:18grounded_sageI guess if you want optimum latency you would probably want to use WebRTC#2024-04-2412:20grounded_sageServer would then just be used for peer discovery#2024-04-2116:29VincentBravo#2024-04-2012:17Eric DvorsakAre you going to propos a talk on Electric at the Heart of Clojure conference?#2024-04-2018:49Dustin Getzsure#2024-04-2021:11Daniel ManilaHas anyone done things with CSS animations in electric? Any examples? I know from react that animating things moving on and off of the dom can be a pain#2024-04-2102:41grounded_sageIt depends what you are doing. Without knowing context. You can add the css class for entry when you add the element and when you are to remove it add the exit and delay the removal of the element with a js timeout. #2024-04-2411:12mattiasWe use such animations, it does make electric app feel a bit more like a regular SPA:
&.active {
background: var(--purple100);
color: var(--purple);
animation: loading 2s infinite;
}#2024-04-2117:06myguidingstarin the electric-starter-app, is there any reason for starting then stopping shadow server in the prod build script here? https://github.com/hyperfiddle/electric-starter-app/blob/18bef64c1dd63d4f297e18dfde7cf5e6e186bdfc/src-build/build.clj#L28C1-L37C26#2024-04-2117:07Dustin Getzbecause shadow-api/release requires a running server, do you know a better way?#2024-04-2117:10myguidingstarI have no idea, just trying to build prod but I have to stop dev repl because otherwise the prod build won't start because dev's shadow already use the port. I just comment out the lines with (shadow-server/start!)and (shadow-server/stop!)and it seems to work, but maybe because I ran the prod build before...#2024-04-2117:14Dustin Getzmaybe you can figure out how to guard it with an if statement#2024-04-2117:56myguidingstarshadow-api/with-runtimelooks promising to wrap around the shadow-api/release#2024-04-2406:16mattiasIt would be cool if you could some day call electric functions defined in other backends. For example, there could be a e/remote-fn function that wraps some remote URL, defines an auth token and which component on the remote gets rendered.
(e/defn MyComponent [arg1 arg2]
(dom/div
(dom/text "this is rendered from local backend")
(new MyComponent2 arg1))
(dom/div
(dom/text "this is rendered from a remote backend")
(new (e/remote-fn 'RemoteComponent "" auth-token) arg1)))
On the remote server, the new router implementation should make it possible to render the output that way. Not sure about the frontend, though, this would mean loading JS from many remotes, which could be tricky security-wise? But if it worked, this would make electric apps very composable in a microservices architecture, or you could just be a "widget provider" for some other electric app.#2024-04-2411:07Dustin Getzwe intend to do this, not this year though#2024-04-2411:08mattiasvery nice 👌#2024-05-2015:52Ben SlessImagine writing your code on an Electric cluster
So cool#2024-04-2604:18grounded_sageI’ve noticed e-for-by causes a lot of flickering in ui components which update their data at a high frequency. I suspect this is something I am missing with respect to how to use it. Has anyone else experienced this?#2024-04-2606:28henrikYeah, this is normal. Dustin knows more, but essentially there are limitations in Electric v2 that makes fors kind of opaque to Electric in terms of calculating how the graph will expand. This is especially noticeable if you have nested for’s, and they rely on data from the server. Last I checked in, Electric v3 is expected to alleviate this considerably.#2024-04-2606:32henrikFor some situations you can kind of get around this by ensuring that all data you will be using throughout the nested fors are available on the client, and that the nested for structure sits client side. This is expected to be an anti-pattern in v3, where it will be able to calculate the entire graph down through the fors ahead of time, and send the required data as a single payload, provided that the control structure for that data also is server-side.#2024-04-2606:44nakkayaI had a similar issue, replacing for with for-by fixed the flickering in my case see [1].
[1] https://app.slack.com/client/T03RZGPFR/C7Q9GSHFV#2024-04-2606:47henrikThat’s a good point. for is equivalent to for-by identity […] , which might cause a lot of re-rendering.#2024-04-2610:58Dustin Getztake a look at how the grid sheet renderer works for a workaround#2024-04-2611:01Dustin Getzalso we have to be very careful in v2 about Pending, which can come from really surprising places (not even we can predict it reliably in v2). catching and silencing pending exceptions (particularly in and around for-by) is the other trick to prevent flicker #2024-04-2717:39grounded_sageThanks for the info! Helpful to know. Atm I am just doing work arounds as well. Looking forward to v3#2024-05-0108:08mattiasProbably hard/impossible to tell, but do we have any estimates for when v3 might come out?#2024-04-2700:01weilooking to build a stateful ui (multi-step login flow or wizards) and i found this thread about statecharts https://clojurians.slack.com/archives/C7Q9GSHFV/p1699957726323019. anyone have additional experience reports since that thread?#2024-04-2708:27henrikPing @U06KD64RX #2024-04-2908:18avocadeHey @U066TMAKS, I have an implementation for clj-statecharts inside electric in production so it definitely works, but there are some tricky parts to keep an eye out for when using electric since everything is reactive. Lots of places where I ended up jumping back and forth from cljs -> clj -> cljs to get a normal control flow.
This is pretty much the setup:
(let [the-fm-atom (atom {})
the-fm (sc/make-machine-with-store the-fm-atom fsm/file-machine-config)
the-fm-state (e/watch the-fm-atom)]
(sc/start-machine the-fm the-fm-atom {:debug true}))
Then I set up an electric fn as a general handler to trigger on state changes (i.e. no use of defined action fns from within the state machine definition, nor entry/exit fns.)#2024-04-2911:31Eric DvorsakIt's still at the "thinking about it" stage for me, but I was looking at doing this using https://fulcrologic.github.io/statecharts/#2024-04-2911:42Eric Dvorsakand someone actually did it in the thread you mentionned https://github.com/roterski/electric-statecharts-demo#2024-04-2914:14avocadeYes I check that demo out before I did my implementation iirc.#2024-04-3011:41Eric Dvorsak@U06KD64RX what are the biggest pros for using statecharts with electric? is there some things that you are able to do with statecharts that you feel like would be worse without in electric?#2024-04-3011:57avocadeWell no, statecharts is a good fit for clojure regardless of electric I'd say. We just use electric everywhere now so I had to learn how to make them be friends.
Most things relating to state will benefit from a state machine imho, as soon as it graduates to more than 2-3 different states. It's just a much better model to architect programs and to aid in thinking, to actually reify otherwise implicit states.#2024-04-3015:36Eric Dvorsakare you using them on the client, server or both?#2024-05-0207:21avocadeIn the client-side context only currently, but should work across.#2024-05-0207:37Eric Dvorsakand do you use some viz tools or text only representation of the statecharts?#2024-05-0208:32avocadeJust the code. It's clear enough for me.#2024-05-0208:36Eric DvorsakI kind of like the idea of being able to viz, especially to use statecharts for docs and QA but it seems like the 2 clj libs for it don't have an easy way#2024-05-0208:52avocadeYeah sometimes nice especially when you want to explain to non-coders.
There is a SaaS that seems pretty nice for drawing statecharts, haven't used it for realz though and can't remember the name.#2024-05-0208:53avocadeQuick google search found it: https://stately.ai/#2024-05-0209:08Eric Dvorsakyes it's the people behind xstate#2024-05-0209:08Eric Dvorsakbut it's JS only and it's quite big#2024-05-0213:13avocadeYes to me it's only interesting as a visualisation tool since it doesn't output anything like clojure.#2024-05-0311:17SigveIf anyone is interested: i made a similar demo using clj-statecharts instead of fulcro/statecharts https://github.com/sigvesn/electric-clj-statecharts-demo#2024-05-0313:59avocadeNice @U01E4ELDYM9#2024-04-2717:38grounded_sageHow can I avoid this issue?
Exception in thread "missionary blk-1" java.lang.Error: Can't process event - observer is not ready.
#2024-04-2717:40nivekuilm/relieve#2024-04-2717:45grounded_sageI have this
(->> (observe-fn input-val)
(m/reductions {} r/pending)
m/stream)
Do I do it before or after reductions?#2024-04-2719:22nivekuilin your case it might be m/latest after reductions#2024-04-2720:56Hendrikm/relieve after m/reductions#2024-04-2913:20Eric Dvorsakwhat does r/pending do?
m/relieve will drop values you might not want that?#2024-04-2913:55grounded_sagenot my code and still yet to deep dive missionary. don't want to drop values#2024-04-2914:16HendrikAs far as I know, r/pending values will be short circuit by electric:
(e/defn foo []
(let [flow-value (new Flow)
(do-something flow-value))) ; if flow value is r/pending or r/failure?, then do-something is not called with that value
#2024-04-2910:42Absolute NegativityI'm seeing weird behavior with Datomic's query:
(e/server
(let [db (ee/watch-db db-conn)
x (dm/q
'[:find ?e .
:where [?e :db/ident]]
db)]
(log/info :VAR x)
(log/info :EXPR (dm/q
'[:find ?e .
:where [?e :db/ident]]
db))))
Whenever db changes, I'd see only :EXPR logged. This also happens if I substitute dm/q with dm/entity or dm/pull . Other operations such as dm/datoms produce expected result (with :VAR logged). What is going on?#2024-04-2910:57xificurCif x results to the same value as it was before electric won't re-run the :VAR log call since the arguments to it didn't change. Can that be the case here?#2024-04-2911:01Absolute NegativityOh silly me. Yes that's it. Thank you.#2024-04-2911:11Absolute NegativityLooks like it's not the case for the original problem I'm struggling with:
(e/server
(log/info :arg state/db uid)
(let [x (some? (seq (d/entity state/db uid)))]
(log/info :VAR x)
(log/info :EXPR (some? (seq (d/entity state/db uid))))))
Here I make some transaction and got the following logs:
:arg datomic.db.Db@4d9e0613 17592186201160
:VAR true
:EXPR true
;; after tx
:arg datomic.db.Db@add94f9 17592186201160
:EXPR false
Do you have any idea?#2024-04-2911:13xificurCI do! d/entity https://ask.datomic.com/index.php/859/equality-on-d-entity-ignores-db?show=859#q859#2024-04-2911:17Absolute NegativityBingo. Thank you so much.#2024-04-2914:38s-olyikes, that's sneaky#2024-05-0104:16NikolaiHappy to announce the second iteration of Krestianstvo | Electric Clojure with the code name Portal. Read more about the concept here https://docs.krestianstvo.org/en/portal, which is now available in Krestianstvo | Electric Clojure.
You can try the new Portals feature in Krestianstvo World Browser
https://e.krestianstvo.org/(krestianstvo.browser!%42rowser)
The source code is here: https://github.com/NikolaySuslov/krestianstvo-electric under the src/krestianstvo. I will soon publish a blog post about that and also will formulate the accumulated questions I stack around during the development, that still need to be fixed.#2024-05-0213:59Eric DvorsakFirst issue I noticed is that it is really laggy once you use an actual server instead of localhost and start paying latency#2024-05-0223:14NikolaiThanks you Eric for trying it out! Yes, I am also experiencing laggy performance. The closest goal of the current development phase is to get an equivalent implementation of the original OSS Croquet VM in Electric. And I found that going from non reactive implementations of Croquet to the mind blowing reactive Electric DSL required to be very accurate in a lot of aspects. For sure, the closer to Electric/Missionary architectural concepts the fewer lags will be.
The current demo application on the server provides a metronome that produces a master tick every 50ms in the form of external messages flow. Plus a mouse on any e/client produces its own external events stamped by the server, which gives more stress. But, metronome doesn't have to be enabled by default, so pausing it (demo worlds contain pause button) will not affect the running Croquet VM. So, any other external messages e.g. mouse clicks, throttled move events will suit for moving forward the virtual time (Future message sends). Saying like: "animation will go, only while you are pressing buttons or moving a mouse". Metronome object just automate that. I have started describing the implementation of the VM here (https://github.com/NikolaySuslov/krestianstvo-electric/blob/dev/src/krestianstvo/docs/krestianstvo.md). Also Croquet VM requires that just every external message should reach its recipient, as the object state is computed locally on every e/client by itself, interpreting incoming messages.#2024-05-2223:49NikolaiI recently updated the code of https://github.com/NikolaySuslov/krestianstvo-electric, to use a lazy server clock. And that architectural change fixes lot of performance issues. You can read more about this here: https://dev.to/nikolaysuslov/krestianstvo-electric-lazy-reflector-for-croquet-vm-4k93
Also, when using the demo locally or from https://e.krestianstvo.org, please open the "Link to world" in new browser windows (with right mouse click), instead new tabs (default browser behaviour). There is still a mess with sync on initial connection (TODO).#2024-05-2313:52refsetThis is really neat! Have you had feedback on this exploration from other Croquet folks (https://croquet.io/advisors/ Croquet Labs people)?#2024-05-2317:38NikolaiThanks!
Yes, I have made an announcement in the Croquet community public discord and have gotten a positive reaction.
Here is the response From Yoshiki Ohshima (Chief Scientist, Co-founder at Croquet Labs)
"This sounds pretty cool. Yes, I think making tick rate adaptable, or rather as you described, just make a tick when it is precisely needed is the way to go. And I like the idea of explicit notion of continuous and discreet time ala FRP or "denotative, continuous-time programming" (DCTP), as Conal Elliott himself would rather call it."
But I should bring to your attention that the Croquet VM at Croquet Labs is the closed source product. And currently the company is in the active phase of developing the https://multisynq.io/ protocol (based on Croquet & Blockchain).
Instead https://krestianstvo.org/ is a fully OSS project and is based on the publicly available implementations of the Croquet VM (Smalltalk, VWF/JS). And what I can say with almost certainty, is that all known implementations of the Croquet VM (including ongoing Multisynq) belong to STT (Simple TeaTime) where Time is discrete, as the full TT (TeaTime) needs Time to be continuous function.
I attached the slide from Croquet - Hedgehog architecture presentation, 2005 (David A. Smith, Andreas Raab, David P. Reed, Alan Kay). You can also read the recent Multisynq Litepaper about Synchronizer Ticks that move time forward (aka Reflector in Croquet) on pp.11-12. That still follows simple TeaTime approach.(https://multisynq.io/downloads/Multisynq%20Litepaper.pdf).
And today in 2024, due to Electric Clojure, there is a hope that for the Croquet application architecture in general (https://en.wikipedia.org/wiki/Croquet_Project), it's dream to get full TeaTime will finally come true!#2024-05-2322:16refsetAh neat! And yeah it's a shame most Croquet work has been closed source until now. But I'm glad to see there's hope yet 🙂
It still amazes me that David Reed more or less described Datomic (i.e. Rich's "epochal time model") in his http://publications.csail.mit.edu/lcs/pubs/pdf/MIT-LCS-TR-205.pdf(!)#2024-05-2223:49NikolaiI recently updated the code of https://github.com/NikolaySuslov/krestianstvo-electric, to use a lazy server clock. And that architectural change fixes lot of performance issues. You can read more about this here: https://dev.to/nikolaysuslov/krestianstvo-electric-lazy-reflector-for-croquet-vm-4k93
Also, when using the demo locally or from https://e.krestianstvo.org, please open the "Link to world" in new browser windows (with right mouse click), instead new tabs (default browser behaviour). There is still a mess with sync on initial connection (TODO).#2024-05-0206:40HendrikDid somebody already try electric with the latest clojure 1.12 Alpha version?
I get this error:
; I cannot resolve `clojure.lang.PersistentArrayMap/createAsIfByAssoc`, maybe it's defined only on the client?
; If `clojure.lang.PersistentArrayMap/createAsIfByAssoc` is supposed to be a macro, you might need to :refer it in the :require-macros clause.
; Evaluation of file ui.cljc failed: class clojure.lang.Compiler$CompilerException
Minimal example to reproduce
(e/defn Main [ring-request]
(e/server
(let [{:keys [foo]} {}])))
I use this electric version
com.hyperfiddle/electric {:mvn/version "v2-alpha-540-ga4699532"}#2024-05-0207:08xificurClooking into this#2024-05-0211:23xificurCI pushed a fix on master. Newer versions of clojure change (break?) macroexpansion on method calls. I guess we'll have more work to do due to the syntactic changes#2024-05-0213:58Dustin GetzDo we need to report this to Alex?#2024-05-0214:21xificurCthe breakage is explicit in https://clojure.atlassian.net/browse/CLJ-2806
> • Macroexpand - stop expanding (Classname/method ...) to dot form
#2024-05-0307:53HendrikIs ist feasible to use the fixed version on master? I guess a lot of other things with respect to v3 development are going on there. If I should not used the master version, would it then be ok for you to backport the fix to v2?#2024-05-0309:04xificurCmaster doesn't contain v3 changes, so yes, using master should work#2024-05-0406:59Eric Dvorsak> https://clojureverse.org/t/signals-vs-streams/9840/8
> Thank you for the article! Could you be so kind and clarify what you mean by
> Signals have an impedance mismatch with isolated discrete effects (without a corresponding undo operation), because backpressure will discard events and corrupt the system state.
@holyjak it's late but my take on it would be that unlike signals (contiuous flows), you can't afford to lose values in a discrete flow because each value matters, there was another explanation somewhere comparing button clicks (discrete event) and mouse pointer position (signal): you can afford to drop many pointer positions and still have a valid system state, but dropping even a single button click could corrupt it#2024-05-0520:51Jakub Holý (HolyJak)Thank you, I think I get that. If backprassure means that a signal drops value and there is a discrete side-effect hooked into the signal then I see how the sentence make sense. Though I do not get how undo affects the situation. Also, I thought that signals are lazy and only produce a value when samples (on a “pull”), which is their way of handling back-pressure (= ignore input changes until asked to produce a value). So clearly hooking a discrete side-effect, that is expected to capture every valu change, into a signal is a no-go. Unless undo operation somehow make this workable, under some circumstances? And what does “isolated” have to do with this?#2024-05-0613:28Dustin Getzimagine the side effect is mounting a dom node, and it comes with a unmount lifecycle method to remove it#2024-05-0613:29Dustin Getzif the signal fluctuates rapidly, it doesn’t matter that much if the dom mounts and unmounts exactly in sync as long as in the long run they end up in a consistent state. i.e., the consumption rate and the production rate are decoupled#2024-05-0613:31Dustin Getzin fact if nobody is looking (e.g. the browser window is minimized) why bother touching the dom at all, just wait for the window to become visible again and jump directly to the final state#2024-05-0711:25HendrikWhat are best practices for dealing with state in an Electric program? How many watches to atoms should I have client side? Or to use missionaries vocabulary: Where should I use ports on the client and how big should I make circles in the graph? Is it reasonable to use them where needed and make them as small as possible or is it better to go with a fulcro-like having one atom at the top level approach? I know that this is broad question, but I like to here your opinions and experiences :)#2024-05-0712:53henrikThis is largely a design decision depending on the program you’re writing. React people like to have state everywhere, Fulcro and Re-frame likes one big state. I’ve found that somewhere between the two is a good spot.
There is absolutely state that doesn’t have anything to do with one another, in which case it’s good to separate them. On the other hand, dividing state too much is a common mistake as well. For example, in a table with sorting, dividing “ascending/descending” and “sorting field” into two different states wouldn’t lead to any benefits.
So, the boring answer is that this is largely a design decision that is up to you. Being unconstrained by the framework having any opinions in the matter means that it’s your job, for better or worse.#2024-05-0713:21Dustin Getzbig cycles in the graph are fine. Consider: the database is an atom, so a simple crud form that reads from the database and then writes to it, is a very large circle#2024-05-0713:23Dustin GetzI do not have an opinion at this time as to encapsulation vs controlled state#2024-05-0807:36Hendrikthanks for your answers :)#2024-05-1009:56danbuneaI used to have a single state a la re-frame (+ subscriptions), and some components would have their own state which was only important for them not the rest of the application.
For electric no client side big state + subscriptions. Now I consider the big state the actual database on server, and I have some sort of material views which act like the subscriptions I had in reframe. Also some components, have component client side state, which is important from a UI point of view but not from the whole application p.o.v.#2024-05-1700:59VincentServer db = "big state" love it#2024-05-0807:51HendrikWhere should I place authorization checks in my electric program?
Given this pseudo example:
(e/defn Example [db]
(when (is-admin (get-user db))
(e/client
(ui/button
(e/fn []
(e/server
; do I have to add an additional check here?
(do-restricted-action)))))))
Here are my thoughts:
If (is-admin ….) is true, then the when branch is mounted. Therefore the server knows that the callback from client is valid. If (is-admin …) is false, then then the when branch is unmounted / not mounted. Therefore the server knows, that a callback from the client at this stage is not expected. So a callback from a malicious client is ignored or an error is thrown …?
Can I rely on this or is there any chance that the server is handling the callback? This would mean, that the 2nd check in the callback is needed.#2024-05-0815:04Dustin GetzServer will ignore messages from a malicious or lagged client which believes the button is active#2024-05-0815:06Dustin Getzso yes, it's fine for auth to be implemented like this#2024-05-0815:18Hendriknice 🙂 that simplifies authorization a lot. I am always impressed of Electric. It makes everything so simple and transparent. Thanks for the great work Dustin :)#2024-05-0816:19Dustin GetzThanks goes to the amazing engineering team I work with!#2024-05-0809:21nakkaya(defn new-goog-history []
(doto (Html5History. js/window)
(.setUseFragment false)
(.setPathPrefix (str js/window.location.protocol
"//"
js/window.location.host
"/app"))
(.setEnabled true)))
(defn path> [^js h]
(m/observe (fn mount [!]
(let [k (goog.events/listen
h EventType.NAVIGATE
(fn [^js e]
(let [token (.-token e)]
(js/console.log "Received token: " token)
(! token))))]
(let [initial-token (.getToken h)]
(js/console.log "Initial token: " initial-token)
(! initial-token))
(fn unmount []
(goog.events/unlistenByKey k))))))
(e/def !history (new-goog-history))
(e/def path (new (m/relieve {} (path> !history))))
(defn navigate! [^js !history href]
(.setToken !history href))
I have been using the above for navigation with no issues, however I needed to stop using fragment and just update the URL so I can redirect user to correct page after login. What is happening is when setUseFragment is false reloading the page returns the wrong token first time around after that navigation works fine. For some reason initial-token contains half the URL after prefix. i.e prefix is 4065-a59d-25f731f40d28/84558bb4-fdb9-4139-a93c-cc01caff0b3d after that subsequent navigation events works fine. For routing I use when using reitit (r/match-by-path router browser/path). Is there something else I am missing?#2024-05-0900:43VincentI'm guessing the setPathPrefix is consuming the token in an unsavory way#2024-05-0900:45VincentI thought .setPathPrefix wanted something like # which is the classical angular "hey i'm an SPA" indicator. choosing dynamic values might add stale state (?)#2024-05-1008:38nakkayaposting here in case anyone else stumbles on the issue, turns out it is the default transformer that is causing the problem. providing my own fixed the issue.
(deftype CustomTransformer []
Object
(createUrl [this token path-prefix location]
(str path-prefix token))
(retrieveToken [this path-prefix location]
(let [path (.-href location)]
(.substr path (count path-prefix)))))#2024-05-1112:56braai engineerI'm using https://www.npmjs.com/package/qrcode-generator package from NPM to generate an SVG tag containing a QR-code in Electric. The Electric fn looks like this:
(ns scratch
(:require [hyperfiddle.electric :as e]
#?(:cljs ["qrcode-generator" :as qr])))
(e/defn QRCode [s]
(e/client
(js/console.log "qrcode input:" (pr-str s))
(let [qr-type-number 0. ;; 0 – 40. 0 = auto-detect type.
error-correction-level \M ;; \L, \M, \Q, \H
svg-cell-size 6
svg-margin 4]
(dom/div
(let [svg-tag (-> (doto (qr. qr-type-number error-correction-level)
(.addData s)
(.make))
(.createSvgTag svg-cell-size svg-margin))]
(set! (.-innerHTML dom/node) svg-tag))))))
It works when used like so,
(dom/div
(QRCode. ""))
But when the argument depends on a value from the server, it seems to generate an invalid or partial QR code, e.g.
(e/server
(let [some-id "1234567890"]
(e/client
(dom/div
(QRCode. (str "" some-id))))))
I suspect the QRCode is mounting with some weird input but not re-rendering the SVG when the value arrives. I tried to catch the Pending exception, but doesn't help. Any ideas?#2024-05-1113:03braai engineerhave confirmed that it only renders the initial value and does not update when the e/fn input changes. What am I doing wrong here? Suspect it has something to do with setting the innerHTML of dom/node which maybe overrides something Electric-related.#2024-05-1113:11braai engineerI found an old https://github.com/hyperfiddle/electric-examples-app/blob/1275f316e68520765446d2c294d4b5ea14cfc4e1/src/wip/js_interop.cljc#L7C1-L27C29 in electric-examples-app, which uses an external library that works for changing e/fn inputs.
Is mine breaking because my set! only runs once on initial load even though svg-tag is a changing value?#2024-05-1113:18braai engineerSwitched to the https://www.npmjs.com/package/@luojianet/qrcode and e/fn looks like this:
(ns scratch
(:require [hyperfiddle.electric :as e]
#?(:cljs ["@luojianet/qrcode" :as qr2]))
(e/defn QRCode2 [value]
(e/client
(dom/div
(let [qrcode (qr2. dom/node (clj->js {:width 128
:height 128
:colorDark "#000000"
:colorLight "#ffffff"
:correctLevel qr2/CorrectLevel.H}))]
((fn [value]
(.clear qrcode)
(.makeCode qrcode value))
value)))))
#2024-05-1311:16braai engineerAh, I didn't notice that. The second one with the trick is from one of your examples.#2024-05-1211:20Nikolas PafitisI'm trying to make a wrapper for franken-ui components in electric, but I seem to not really get the dom stuff properly in electric. Here's my wrapper for the accordion component:
(e/defn AccordionTitle [title]
(e/client
(dom/a (dom/props {:class "uk-accordion-title"
:href true})
(dom/span (dom/text title))
(dom/span (dom/props {:class "uk-accordion-icon"
:uk-icon "icon: chevron-down; ratio: 0.8"})))))
(e/defn AccordionContent [content]
(e/client
(dom/div (dom/props {:class "uk-accordion-content"})
(dom/p (dom/text content)))))
(e/defn AccordionItem [& items]
(e/client
(dom/li
(e/for [item items]
item))))
(e/defn Accordion [& items]
(e/client
(dom/ul (dom/props {"uk-accordion" true
:class "uk-accordion"})
(e/for [item items]
item))))
and this is an example usage:
(ui/Accordion.
(ui/AccordionItem.
(ui/AccordionTitle. "Item 1")
(ui/AccordionContent. "Lorem Ipsum")))
But it doesn't render properly. Inspecting the resulting html looks like the appended screenshot. As you can see the elements are not nested as I would expect.
Meanwhile this renders properly:
(dom/ul
(dom/props {"uk-accordion" true
:class "uk-accordion"})
(dom/li (dom/props {:class "uk-open"})
(dom/a (dom/props {:class "uk-accordion-title"
:href true})
(dom/span (dom/text "Item 1"))
(dom/span (dom/props {:class "uk-accordion-icon"
:uk-icon "icon: chevron-down; ratio: 0.8"})))
(dom/div (dom/props {:class "uk-accordion-content"})
(dom/p (dom/text "Lorem Ipsum"))))
(dom/li
(dom/a (dom/props {:class "uk-accordion-title"
:href true})
(dom/span (dom/text "Item 2"))
(dom/span (dom/props {:class "uk-accordion-icon"
:uk-icon "icon: chevron-down; ratio: 0.8"})))
(dom/div (dom/props {:class "uk-accordion-content"})
(dom/p (dom/text "Lorem Ipsum")))))
I'm using the following electric version:
com.hyperfiddle/electric {:mvn/version "v2-alpha-540-ga4699532"}
Am I missing something obvious? What are best practices to create reusable components?#2024-05-1211:48Dustin Getzon mobile but use electric lambda to define fragments, the rendering evaluation model is effectful/imperative so your fragments as defined here are going to render inside out#2024-05-1212:08Nikolas PafitisI'd appreciate if you can link me an example when you can.#2024-05-1212:46Nikolas PafitisAlso would this be more ergonomic and idiomatic API for use in electric?
(e/defn AccordionTitle [title]
(e/client
(dom/a (dom/props {:class "uk-accordion-title"
:href true})
(dom/span (dom/text title))
(dom/span (dom/props {:class "uk-accordion-icon"
:uk-icon "icon: chevron-down; ratio: 0.8"})))))
(e/defn AccordionContent [content]
(e/client
(dom/div (dom/props {:class "uk-accordion-content"})
(dom/text content))))
(e/defn AccordionItem [{:keys [title content]}]
(e/client
(dom/li
(AccordionTitle. title)
(AccordionContent. content))))
(e/defn Accordion [items]
(e/client
(dom/ul (dom/props {"uk-accordion" true
:class "uk-accordion"})
(e/for [item items]
(AccordionItem. item)))))
and usage:
(ui/Accordion.
[{:title "Item 1"
:content "Lorem Ipsum"}
{:title "Item 2"
:content "Lorem Ipsum"}])
Renders as expected, although it feels it might be less flexible from the user perspective than my initial approach.#2024-05-1213:42Nikolas Pafitis@U09K620SG I played around as per your suggestion and managed to get it working like so:
(e/defn AccordionTitle [title]
(e/client
(dom/a (dom/props {:class "uk-accordion-title"
:href true})
(dom/span (dom/text title))
(dom/span (dom/props {:class "uk-accordion-icon"
:uk-icon "icon: chevron-down; ratio: 0.8"})))))
(e/defn AccordionContent [content]
(e/client
(dom/div (dom/props {:class "uk-accordion-content"})
(dom/p (dom/text content)))))
(e/defn AccordionItem [title content]
(e/client
(dom/li
(title.)
(content.))))
(e/defn Accordion [body]
(e/client
(dom/ul (dom/props {"uk-accordion" true
:class "uk-accordion"})
(body.))))
and usage:
(ui/Accordion.
(dom/div
(e/fn []
(ui/AccordionItem.
(e/fn [] (ui/AccordionTitle. "Item 1"))
(e/fn [] (ui/AccordionContent. "Lorem Ipsum")))
(ui/AccordionItem.
(e/fn [] (ui/AccordionTitle. "Item 1"))
(e/fn [] (ui/AccordionContent. "Lorem Ipsum"))))))
API feels a little hacky to the user, Maybe my components should be macros so that the e/fn is not required by the user and it's added by the macro. Maybe a defcomponent can be done for components with a body, I don't know.#2024-05-1213:43Nikolas PafitisDoes anybody have any experience with creating modular components on electric? What are best practices.#2024-05-1214:14Nikolas Pafitis@U09K620SG I think i get what you say, if i try to log dom/div it returns nil. dom/div is a side-effectful procedure that returns nothing, and instructs the browser to create an element and attach it to the parent element. Is that correct?#2024-05-1214:21Dustin Getzyes that’s correct#2024-05-1214:21Dustin Getzthere is also e/partial#2024-05-1214:22Dustin Getzwe have not yet begun to work on a hiccup like authoring DSL, we are deeply committed to a strategy of working out powerful and correct semantics first before moving onto a friendly authoring syntax #2024-05-1214:23Dustin Getzas you mention, you can use macros as a stopgap to eliminate the continuation boilerplate#2024-05-1214:40Nikolas Pafitis@U09K620SG I understand. I think a more functional and declarative approach (might be hiccup DSL, might be just s-exprs) that would return instructions on how to render on the browser would be crucial. I can see scenarios where dom/props attaches props to a different element than the element one would expect etc. I think it would make the API more composable. Is this something that is in the roadmap, or something that is not really planned to be implemented?#2024-05-1214:53Nikolas PafitisAlso I'd like to ask, is it practically feasible if i give it a shot to implement something like that as an external library or is this something that would require changes to electric code?#2024-05-1215:02Dustin Getzsee discussion here: https://github.com/hyperfiddle/electric/issues/32#2024-05-1215:07Dustin Getzthe roadmap sequence is: electric v3 -> ui5 (callback free editable forms) -> editable datagrids with high performance -> finally syntax #2024-05-1215:07Dustin Getzif we skip ahead we are very unlikely to produce an answer that is compatible with the skipped steps#2024-05-1215:34Nikolas PafitisI see#2024-05-1222:32Nikolas PafitisIs it possible to create an element with an html id?#2024-05-1300:35Dustin Getzdid you try using dom/props?#2024-05-1222:33Nikolas PafitisAlso, on a tangent, it might be useful if the element functions like dom/div return a reference of the DOM node rather than just nil#2024-05-1300:33Dustin Getzdom/node is bound and available inside the body continuation of all electric dom elements #2024-05-1300:34Dustin Getzdom/div returns its final child, so to return the node you can write (dom/div dom/node)#2024-05-1300:39Dustin Getzelectric-dom is like 300 LOC, you should probably read the source#2024-05-1307:59JAtkins(This is very likely a common question but I couldn't find any useful ref outside of https://clojurians.slack.com/archives/C7Q9GSHFV/p1688154851988839 - feel free to just tell me I'm stupid and missed prior art 🙂)
I too am messing with Codemirror and would like to leverage (e/server ...) to get completions from the server. It's a very similar situation to dom listeners, except that I care about the return type of my thunks and I'd like them to block the dom (cm) callback waiting for the result. Is there an easy way to boot a temporary electric runtime for just the lifetime of a plain JS callback? I've tried various things, including just (m/? (new server-info data)) and (new server-info data) (where server-info is an e/fn and the callsite is just a plain js fn) but probably obviously this doesn't work.#2024-05-1308:05JAtkinsI also tried a couple variations of code similar to ui4/button but didn’t figure out how to return the result in a blocking way#2024-05-1309:35Dustin Getzcan you post pseudo code for this so we can understand the goal#2024-05-1313:33JAtkinsBroadly, it’s
(cc/defn get-completions [^js context]
;; need something that does this
(let [server-data (js->clj context]
(clj->js (e/server (server-get-completions server-data)))))#2024-05-1314:44Dustin GetzI would look for an async API and then you can use an atom or missionary port to send values from electric into the clojure callback#2024-05-1312:46siddharth yadavwhere would electric and missionary fit in the following graph? taken from https://webperso.info.ucl.ac.be/~pvr/VanRoyChapter.pdf#2024-05-1318:47Dustin Getzelectric is Continuous synchronous programming maybe? (yellow middle)#2024-05-1709:34mattiasIn the electric-fiddle router implementation we have:
(r/router (r/HTML5-History.)
(let [route (or (ffirst r/route) `(Index)) ; route looks like {(f args) nil} or nil
[f & args] route]
(case f
`Index (Index.)
(r/focus [route]
(e/apply (get hf/pages f NotFoundPage) args)))))
Why is the case f necessary when (r/router (r/HTML5-History.)
(let [route (or (ffirst r/route) `(Index)) ; route looks like {(f args) nil} or nil
[f & args] route]
(r/focus [route]
(e/apply (get hf/pages f NotFoundPage) args))))#2024-05-1713:27Geoffrey GaillardBecause Index’s job is to list all routes from hf/pages.
hf/pages doesn’t contain an entry for Index. Otherwise Index would render a link to itself. We thought it might be confusing to newcomers.#2024-05-1713:52grounded_sageperformance wise how does it compare doing the (e/throttle 16) vs using a RAF id and directly calling request animation frame?
https://github.com/hyperfiddle/electric/blob/a40ee67444d739342a8ec2a8715a84c8295f4b39/src/hyperfiddle/electric_ui4.cljc#L324#2024-05-1714:02grounded_sageI am assuming the difference is very small I am just curious if anyone has tested it#2024-05-1714:46Dustin Getzi dont think we tried i think we were being lazy#2024-05-1715:00grounded_sagenp#2024-05-1720:45braai engineerI'm making a toy "text-based" game in Electric where time matters. What's the best way to handle time-based ticks where every few 100ms (e.g. 500ms) I get a new "frame" with the time delta, so I can apply next-state based on time that elapsed?
Currently I have in my entrypoint:
#?(:cljs (defonce !state (atom initial-state)))
#?(:cljs (defonce !ticker (atom nil)))
#?(:cljs (defonce !time (atom (js/Date.now))))
(defn next-state [state]
... ;; do things
state')
(e/defn Game []
(e/client
(swap! !ticker (fn [v] (if (nil? v)
(js/setInterval (fn []
(reset! !time (js/Date.now))) ;; here we would apply next-state to !state to update views.
1000))))
(e/on-unmount (fn []
(js/clearInterval @!ticker)
(reset! !ticker nil)))
In JS land I would usually do requestAnimationFrame (which I think Electric does under the covers?) and apply the time delta. For this application, don't need ms-precision, but would still like an accurate timer to avoid drift.
Any game Electric game examples out there? Would like to do some WebGL rendering for the player as well to spice things up.#2024-05-1720:48Hendrik(handle-frame ( round-to 0.1 e/clock))
roughly something like that.
answering from mobile. therefore i can not test
#2024-05-1720:56Dustin Getzyes start with e/system-clock-ms which already uses raf under the hood on browser#2024-05-1720:57Dustin Getzit also pauses when you minimize or hide the tab#2024-05-1720:58Dustin Getzto get the flow of deltas you can use m/reduce with - #2024-05-1721:00braai engineerWhat if I want the game to keep running even in background, but not render?
Do I have to do that on the server?
How about dis:
(defn handle-frame! [!state time]
(let [state @!state
last-time (:time state)
elapsed-time-ms (- time last-time)]
(when (> elapsed-time-ms 0)
(let [state' (next-state state elapsed-time-ms)]
(swap! !state update-in [:time] time)))))
; (do-more-game-things))))
(e/defn Game []
(e/client
(let [time (e/system-time-secs)]
(handle-frame! !state time))
(dom/div
(dom/text "State:" (pr-str state))) ...
#2024-05-1721:02Dustin Getzwhy do you want the game to not pause? i think you want a pause followed by a very large delta t#2024-05-1721:02braai engineerOK in this case pausing is fine.#2024-05-1721:04nivekuilyou should separate logic and rendering clocks. see https://gafferongames.com/post/fix_your_timestep/#2024-05-1721:09braai engineerHmm in component,
(e/defn Game []
(e/client
(let [time e/system-time-secs]
(dom/div (dom/text "Time: " (pr-str time)) ;; this renders continuously with sub-second precision? should I round this?
...
#2024-05-1721:15braai engineerIf I round down e/system-time-secs or e/system-time-ms to 1s, isn't that consuming a bunch of CPU rounding on every frame, even though it's not re-rendering? E.g.
(let [time (* 1000 (clojure.math/round (/ e/system-time-ms 1000.0)))
state (e/watch !state)]
(handle-frame! !state time)
I would have expected that e/system-time-secs would already be rounded down to 1000ms, but OK I see why it's a float.#2024-05-1721:24Dustin Getzit is rounding every frame (there is also DAG overhead every frame) but if you think about it, your computer can do a trillion flops and render video games in realtime, and e/system-time-ms is properly backpressured (only consuming idle cpu as per raf scheduling) so in principle you're only consuming idle flops#2024-05-1721:24Dustin Getzif you see a perf issue (the DAG overhead could maybe be significant in large DAGs) we want to know#2024-05-1721:26Dustin Getzthere can also be accidental network overhead in electric v2 now that i think about it#2024-05-1810:35braai engineerIs there a way to run Electric on the client only? I.e. so I can host it on S3 as static JS with no websocket connection.#2024-05-1813:19xificurCWe have a test entrypoint for a single peer here, as well as def/defn that are single peer https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_local_def.cljc#2024-05-1823:43Vincentclient only means ajax everything?#2024-05-1909:31braai engineerNo sever required, i.e. only compile cljs.#2024-05-1918:05VincentCool. May I ask what you're using it for?#2024-05-2021:03braai engineerBehold, BraaiSim (alpha): The Braai Simulator built in 2 days with Electric Clojure.
Braai means barbecue in Afrikaans, my native tongue. Remember to turn the meat & stay hydrated!
Progress:
🤳 Game is now mobile-friendly
♻️ Tap to turn food on the grid. The core mechanic is all about timing and memory (“did they ask for medium-rare or well-done?”)
🥵 Turning food bare-handed over a hot fire reduces HP. Eventually you die of thirst, 3rd degree burns or food poisoning if you don’t cook the chicken properly.
🍺 Sipping too much beer makes the screen wobble and distorts cooking time
Coming Soon:
🏆 Earn points for serving perfectly-cooked food & win the title of braaimaster
⏳ Tme bar next to each food item
🍔 More foods 🐓, 🌽, 🍄, 🍆
🪵 Control heat by adding wood to the fire
🧤 Upgrade your braai tools with gloves, crappy red tongs & proper stainless steel tongs
🥃 Brandy & coke adds HP but you will randomly drop food & customers swap around
🍳 Upgrade your braai for a bigger grid.
Cheers 🍻
Twitter Link where I tagged Electric Clojure: https://x.com/petrustheron/status/1792660925624840489?s=46
Will post a link when it’s ready to play.#2024-05-2021:05braai engineerBraai: https://en.m.wikipedia.org/wiki/Regional_variations_of_barbecue#South_Africa#2024-05-2114:18Dustin Getzlol#2024-05-2121:00VincentSlammin'.#2024-05-2321:16braai engineerBraaiSim progress update:
:male-technologist: Mobile touch events & zIndex are the hardest parts of making a game in Electric Clojure.
🪵 Earn $Braaicoin (top-right) by serving food the way guests like it. It’s still too hard, so will add food cooking bars.
❤️ Nicer status bars for health, hydration & heat
🔥Tap wood to stoke the fire (adds heat)
:frame_with_picture: Transparency for some images
:knife_fork_plate: Paper plates for serving food
:mending_heart: Player now heals after touching hot-food with bare hands.
Will share link to channel when it’s more playable.#2024-05-2502:30Vincentcooking bars ... braai yeah!#2024-05-1822:48Nikolas PafitisI have just finished and published my wrapper https://github.com/conjurernix/electric-franken to use https://www.franken-ui.dev/ in electric. I have to say, that after delving deeper with electric's DOM api, I grew to like it alot, maybe even more than hiccup-like APIs. I find it extremely flexible and composable.#2024-05-1822:57Dustin Getzwow i didn’t understand what you were working on, i will try it out this week! thank you for this contribution!#2024-05-1822:58Dustin Getzare you building some specific app with this?#2024-05-1823:29joshchothis is amazing! i wanted to try shadcn in electric but didn’t realize franken-ui existed not to mention you made a library, tysm#2024-05-1823:44Nikolas Pafitis@U09K620SG I am planning to actually yes#2024-05-1911:39Nikolas Pafitis@U09K620SG In the beginning I was trying to be too smart, and was trying to do a "Rich" wrapper with an API in what you find from https://mui.com/ because that's what I was used to. But then I realised that you can effectfully modify the component using style and option modifiers, and now I see that it's much more flexible both in UX and in the development of the wrapper. Supporting a new component option requires simply adding a helper function that modifies the encapsulating element.
Edit: And it stills feel declarative from the user's perspective.#2024-05-1915:50Dustin GetzThanks for outlining the evolution of your thought process (and appreciate you updating the ticket as well for the next person to see). Perhaps we write a blog post about the effectful style, given that we won't be investing in a declarative syntax frontend any time soon#2024-05-1915:53Dustin Getzquick first glance feedback - we highly recommend to capitalize Electric function names and use lowercase names for macros and native clojure functions, otherwise it's hard to remember which call convention to use (`new` vs regular apply) - and there's no type system to help, and we haven't figured out yet how to get a good error message for this mistake#2024-05-1920:29Nikolas PafitisIn the library i define only macros. Macros that are components, i did uppercase and macros that are helpers are lowercase. That way you sort of distinguish which one is a component. Also there's some overlap, there's the Drop component and the drop helper, where the Drop component is just a div with the drop modifier with no arguments. But could change it if not idiomatic.#2024-05-2021:03braai engineerBehold, BraaiSim (alpha): The Braai Simulator built in 2 days with Electric Clojure.
Braai means barbecue in Afrikaans, my native tongue. Remember to turn the meat & stay hydrated!
Progress:
🤳 Game is now mobile-friendly
♻️ Tap to turn food on the grid. The core mechanic is all about timing and memory (“did they ask for medium-rare or well-done?”)
🥵 Turning food bare-handed over a hot fire reduces HP. Eventually you die of thirst, 3rd degree burns or food poisoning if you don’t cook the chicken properly.
🍺 Sipping too much beer makes the screen wobble and distorts cooking time
Coming Soon:
🏆 Earn points for serving perfectly-cooked food & win the title of braaimaster
⏳ Tme bar next to each food item
🍔 More foods 🐓, 🌽, 🍄, 🍆
🪵 Control heat by adding wood to the fire
🧤 Upgrade your braai tools with gloves, crappy red tongs & proper stainless steel tongs
🥃 Brandy & coke adds HP but you will randomly drop food & customers swap around
🍳 Upgrade your braai for a bigger grid.
Cheers 🍻
Twitter Link where I tagged Electric Clojure: https://x.com/petrustheron/status/1792660925624840489?s=46
Will post a link when it’s ready to play.#2024-05-2014:12nakkayaI am trying to figure out what would be the most idiomatic way to achieve the following, I have client side atom that tracks how much storage a user is consuming this value is loaded from the server on startup and displayed on the UI,
(e/def !state (atom (e/server
(e/offload
#(build-state)))))
then when a new file is uploaded I just increment the counter and the UI automatically picks up the changes.
(swap! !state update-in [:events :storage] + (.-size file))
problem is when a file is deleted I need to make a query to the db in order to get the file size then subtract it from the count.
(let [size (e/server
(let [file (db/get-file file-id)]
(db/get-file-size file)))]
(swap! !state update-in [:events :storage] - size))
this does not work since by the time size is calculated swap! already ran. What would be the best way to structure these updates?#2024-05-2014:33xificurCswap! depends on size, it cannot run before it's calculated#2024-05-2014:41nakkayaIf I put a print statement right before swap! nothing is printed to console server side.
(let [size (e/server
(let [file (db/get-file file-id)]
(db/get-file-size file)))]
(e/server (println "to be removed" size))
(swap! !state update-in [:events :storage] - size))
but if I do,
(let [size (e/server
(let [file (db/get-file file-id)]
(db/get-file-size file)))]
(e/server (println "Hello")))
I do get the output any reference to size breaks the flow. As a side note this section is triggered from within an
(dom/on "click" (e/fn [_] ....))
if that makes any difference. @U09FL65DK#2024-05-2015:02xificurCcan you share the entire dom/on call?#2024-05-2015:37nakkayathat's pretty much it,
(h/on-click
(let [size (e/server
(let [file (db/get-file file-id)
size (db/get-file-size file)]
(db/delete-file file-id)
(println "remove" size)
size))]
(e/server (println "to be removed" size))
(swap! !state update-in [:events :storage] - size)
(event)))
h/on-click is just,
(defmacro on-click [& body]
`(dom/on "click" (e/fn [_] ~@body)))
and event is just a function that triggers an update via an atom/watch for a different part of the UI.
(fn [] (swap! e inc))
@U09FL65DK#2024-05-2018:34xificurCdon't see anything suspicious. Could you try to wrap the swap! in a prn to double check it doesn't run on the client? (prn :swap-size! (swap! !state ..))#2024-05-2018:43nakkayano change, nothing is printed to the browser console, I can tell event after swap works cause table updates but other two statements are no-ops.#2024-05-2018:56xificurCI've spent way too long on v3 at this point so I might be looking at this with the wrong semantics in mind. Can you try to rotate the colors a bit?
(h/on-click
(e/server
(let [file (db/get-file file-id)
size (db/get-file-size file)]
(db/delete-file file-id)
(e/client
(prn :swap-size! (swap! !state update-in [:events :storage] - size))
(event)))))#2024-05-2018:57xificurCalso, if (event) should run after everything else finished you should guard it as (case (prn :swap-size! (swap! ..)) (event)), this sequences the effects#2024-05-2019:14nakkayathanks this worked, whats the difference between initial and this version? @U09FL65DK#2024-05-2019:19xificurCI'm not quite sure unless I tried to repro locally. I know (e/client (let [size (e/server ..) is safe in v3 but suspected it might not work as expected in v2. Are you using this pattern elsewhere successfully?#2024-05-2019:22xificurCdom/on has tricky semantics, it unmounts the code after seeing a non-Pending value. I suspect your code manages to sneak one through somehow. We're in the process of designing a better and more robust event handling API as we speak#2024-05-2019:34nakkaya> Are you using this pattern elsewhere successfully?
No not from on click this is the first time I needed it.#2024-05-2017:41Tiago LuchiniFolks, I am migrating an app from Electric v2-alpha-422-g96e5c0a5 to v2-alpha-540-ga4699532 . Two weird behaviors that I am dealing with but would make me believe I have done something wrong:
1. Any component (`e/defn`) I had without an e/client or e/server failed. I had a series of client components that inherited e/client from higher up in the call stack.
2. Small changes to the dom do hot-reload fine but anything more "drastic" (like adding/removing a dom element) makes the reactor fail. The server throws a Websocket handler failure. with code 1011. Stopping and restarting the server fixes the issue.
Any thoughts?#2024-05-2017:48Dustin GetzThe siting rules changed in 536, see https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md. This probably explains #1#2024-05-2017:48Dustin GetzIt probably also explains #2#2024-05-2017:49Dustin Getzit could also be the stricter definition order breaking change in 536#2024-05-2017:49Dustin Getzit can also be the breaking change that e/defn is no longer allowed in a cljs file, it needs to be a cljc file#2024-05-2018:09Tiago LuchiniThanks @U09K620SG. 536 was a big one 🙂#2024-05-2018:10Tiago LuchiniI wonder if the way I am starting the server is short-circuiting the recompilation of the server. Instead of the setup you have on the starter app, I've plugged the jetty server onto my repl-reloaded setup via integrant#2024-05-2018:12Tiago LuchiniUltimately that renders me with e/boot-server and e/boot-client in different places. Would that affect the server recompilation? (I clearly "fix" the server by re-evaluating it after a change so the breaking changes feel correct)#2024-05-2018:15Dustin GetzIt is hard to say, step 1 would be to prove that it works outside the custom setup#2024-05-2018:20Tiago LuchiniIn your workflow, do you save and re-eval?#2024-05-2018:22Tiago LuchiniKnowing the IC difference introduced by 536, it makes sense that I might just have "learned" a lazy workflow: saving changes and letting shadow-cljs pick up the slack and go through hot reload. On my setup, I can also fix my issue be reavaluating my namespace.#2024-05-2018:31Dustin Getzi just save, and shadow will reload jvm (there is a shadow hook facilitating reload). if you eval jvm only but not cljs you’ll get mismatched electric programs#2024-05-2018:33Tiago LuchiniGot it! It is my setup then. I've got shadow on a separate jvm from my repl-reloaded. I will work on changing it.#2024-05-2018:33Tiago LuchiniThanks for the nice work as usual... IC is a great addition!!!#2024-05-2018:38xificurCthe changelog also notes there's a new shadow build hook, if you have your own setup you'll have to add it, otherwise the server side won't recompile#2024-05-2018:40Tiago LuchiniIt makes sense now @U09FL65DK. This is great!#2024-05-2020:10Tiago LuchiniUnifying shadow via integrant and bringing it under the same jvm solved all issues and is working wonderfully and made the whole setup safer, clearer and lighter. Thank you.#2024-05-2020:10Dustin Getzexcellent, are you using the "save file" workflow exclusively or is integrant doing something?#2024-05-2020:13Tiago LuchiniMostly using the "save file" workflow. Integrant becomes mostly a holder for config and server-side dependencies.... repl-reloaded per se is not a recurring need if the "save file" workflow works#2024-05-2110:01braai engineerHas anyone tried to package up an Electric Clojure app into a native mobile app using a WebView? In my use-case, I only need local rendering (no server required) i.e. only Cljs, but Clj would be a bonus.
Reason I want to do this is because I'm having a hard time playing audio in mobile browser.#2024-05-2120:20Dustin Getzi think they did it here: https://clojurians.slack.com/archives/C7Q9GSHFV/p1683502145767229#2024-05-2218:30Vincentusing the latest git/sha a40ee67444d739342a8ec2a8715a84c8295f4b39 i'm under the expectation that incremental compilation will work out of the box, is this misdirected#2024-05-2310:10braai engineerUpdate: Fixed with DataScript externs.
Any ideas why DataScript queries in an Electric app would work in a dev build, but break in a production build with no errors? I can inspect the DataScript DB and the data is there, but queries just come up empty.
Not sure if this is related to Electric or DataScript cljs prod build.#2024-05-2310:18braai engineerIt was externs. It's always externs. In shadow-cljs.edn:
:compiler-options {:externs ["datascript/externs.js"]}#2024-05-2315:55JAtkinsI suspect I've found an issue with the electric compiler.
web | #error {
web | :cause "Index 2 out of bounds for length 2"
web | :via
web | [{:type java.lang.ArrayIndexOutOfBoundsException
web | :message "Index 2 out of bounds for length 2"
web | :at [clojure.lang.RT aget "RT.java" 2377]}]
web | :trace
web | [[clojure.lang.RT aget "RT.java" 2377]
web | [hyperfiddle.electric.impl.runtime$eval_change_inst invokeStatic "runtime.cljc" 830]
web | [hyperfiddle.electric.impl.runtime$eval_change_inst invoke "runtime.cljc" 825]
web | [clojure.lang.PersistentArrayMap kvreduce "PersistentArrayMap.java" 429]
web | [clojure.core$fn__8525 invokeStatic "core.clj" 6908]
web | [clojure.core$fn__8525 invoke "core.clj" 6888]
web | [clojure.core.protocols$fn__8257$G__8252__8266 invoke "protocols.clj" 175]
web | [clojure.core$reduce_kv invokeStatic "core.clj" 6919]
web | [clojure.core$reduce_kv invoke "core.clj" 6910]
web | [hyperfiddle.electric.impl.runtime$parse_event invokeStatic "runtime.cljc" 848]
web | [hyperfiddle.electric.impl.runtime$parse_event invoke "runtime.cljc" 843]
web | [clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
web | [clojure.core$reduce invokeStatic "core.clj" 6885]
web | [clojure.core$reduce invoke "core.clj" 6868]
web | [clojure.core$partial$fn__5910 invoke "core.clj" 2648]
web | [missionary.impl.Util apply "Util.java" 29]
web | [missionary.impl.Sample transfer "Sample.java" 84]
web | [missionary.impl.Sample$Process deref "Sample.java" 34]
web | [missionary.impl.Reactor pull "Reactor.java" 183]
web | [missionary.impl.Reactor touch "Reactor.java" 222]
web | [missionary.impl.Reactor propagate "Reactor.java" 242]
web | [missionary.impl.Reactor event "Reactor.java" 398]
web | [missionary.impl.Reactor$1 invoke "Reactor.java" 480]
web | [missionary.impl.Relieve$1 invoke "Relieve.java" 86]
web | [missionary.impl.Sample$3 invoke "Sample.java" 146]
web | [missionary.impl.Observe$1 invoke "Observe.java" 74]
web | [clojure.core$comp$fn__5876 invoke "core.clj" 2586]
web | [hyperfiddle.electric_ring_adapter$electric_ws_handler$on_message__128110 invoke "electric_ring_adapter.clj" 175]
web | [clojure.lang.AFn applyToHelper "AFn.java" 156]
web | [clojure.lang.AFn applyTo "AFn.java" 144]
web | [clojure.core$apply invokeStatic "core.clj" 669]
(e/defn ViewAccounts
[accounts show-org-name?]
(e/client
(let [!search-str (atom "")
search-str (e/watch !search-str)
faccounts (e/server (filter (fn [a]
(or (= (:account/name a) search-str)
(= (:account/email a) search-str)))
accounts))])))
The issue is that search-str is being accessed twice and causing the index out of bounds. If I introduce a (let [ss search-str] ... around the cc/fn, it doesn't cause errors#2024-05-2319:13grounded_sageHow do you use (e/def db) across different namespaces and components?
I keep getting unbound electric var#2024-05-2319:20henrikTypically you bind it during some boot sequence of the app, after which it is available on the var in all Electric functions launched from that closure.
You have to pass it in manually to Clojure functions though, as the binding is only valid I. Electric land.#2024-05-2319:21grounded_sageI've seemed to have solved it by moving (e/def db) to another namespace and beinging that in :refer [db] to the namespaces I want it in and doing a binding in each namespace. But this doesn't seem right.#2024-05-2319:23henrikIf you’re hooking it up statically on the var, it should be available everywhere. Rebinding per NS shouldn’t be necessary.#2024-05-2319:24Dustin Getzcheck client/server coloring, maybe resolving server var on client?#2024-05-2319:24Dustin Getzit's not automatic like let#2024-05-2319:24grounded_sageI am doing (binding [db (e/watch conn)] at the root. But everytime I pass db to an Electric function or try and use it in another namespace it says it's unbound.#2024-05-2319:26grounded_sageYea I thought about client/server coloring and thought I might be making an error there but I still am not understanding something#2024-05-2319:26henrikIf you’re binding it in server scope, are you also using it from server scope subsequently? Ie., coloring as Dustin mentions.#2024-05-2319:28grounded_sagens-1
(e/server (binding db ... (e/client)))
what is the best way for me to get db into
ns-2
(e/defn MyFN [ ] (let [result (e/server (d/q .. db)] (e/client result))
#2024-05-2319:29Dustin GetzTo effectively help we need to see a complete and small snippet, can you reproduce it in the starter app?#2024-05-2319:30grounded_sageI can do that. It just seems like a straightforward thing that other people would be doing that I misunderstand. So I thought I would get a quick answer.#2024-05-2319:30henrikWe have an NS with globals like (e/def db) etc. Those are bound at the app root. Other NSes can then require the globals NS.#2024-05-2319:31grounded_sageI probably just have some sleep and then it works tomorrow. I can create a repro if not.#2024-05-2320:27braai engineerPerformance question: better to pass a DataScript DB ID to an Electric fn and query a DB entity in the function (against (binding [db (e/watch conn)] ...) bound at a top-level function), or better to hydrate the entity in top-level function and pass as an argument into the top-level component? Same number of DB queries, but I imagine fewer queries is better?#2024-05-2320:30Dustin Getzbroadly i recommend keeping things serializable (for client/server transfer) which means passing around entity ids wherever possible#2024-05-2320:32Dustin Getzwhen your db changes, all queries are gonna get touched regardless if the database doesn't offer streaming query subscriptions#2024-05-2320:32Dustin Getzyou can use missionary to throttle at that point if you are facing some large transaction volume or something#2024-05-2320:33braai engineerMakes sense. In my case (BraaiSim), it's all Cljs (for now).#2024-05-2411:44braai engineerThis is not Electric-specific, but is there a good way to preload images for later display in Electric? Ideally not at the same time as the images currently needed, so might need to wait for other images to render (I guess I can tap into the images' (dom/on "load" ...) event? Maybe cache images using :visibility "hidden"?#2024-05-2416:57Geoffrey Gaillard<img> has a https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img#loading attribute.#2024-05-2417:00Geoffrey Gaillard:visibility "hidden" might work.
Also <https://web.dev/articles/preload-responsive-images#preload%7C<link rel="preload">>
But not sure how to "preload" AND "defer" image loading at the same time.#2024-05-2417:05Geoffrey GaillardAlso new Image("url") will load the image immediately, even if not added to the DOM.#2024-05-2502:25VincentIs there a way to do keydown and keyup on the same (dom/on) obj?#2024-05-2506:44Geoffrey GaillardNot today. For now you'll need two dom/on calls.
Have a look at dom/Focused and dom/Hovered. It's not based on dom/on but it might still help.#2024-05-2717:09Vincentokay cool#2024-05-2613:47jaihindhreddyFrom what I can tell, the client and server output by the Electric-compiler are tightly coupled, and the server rejects conns from old clients.
If a user is filling-out a long form, and a production deployment of the electric app happens in the middle, they lose the state unless we program a way around it.
Is this correct, and if so, what would be some reasonable ways of getting around this?
Edit: A more succinct version of my question would be "how do I deploy an electric app a few times an hour without annoying users with random refreshes?"#2024-05-2613:54Dustin Getzthis is correct and you'll need to deal with it in userland for now, in theory "over the air updates" are possible#2024-05-2613:55Dustin Getzyou can also add ops complexity to keep old server versions running until all clients disconnect#2024-05-2613:56jaihindhreddyYeah, that's how I was thinking it could be done.
Keeping old server versions running, but letting them know that they're old, which in-turn results in the older clients showing a "newer version available, please refresh" floating modal somewhere.#2024-05-2613:57Dustin Getz"user is filling-out a long form" -> for the record, electric makes possible a form architecture where all user interactions are immediately streamed to the server as they type them, where the state can be made durable even before committing to a database#2024-05-2613:58jaihindhreddyYeah, long form was not the best example to ask my question.#2024-05-2703:51grounded_sageI have had similar inquiries around this line of questioning. I love how Electric is like well we can just solve that because it’s easy now.
So then the question I have. What are some use cases, scenarios where this does fall over. Where you do have to handle persisting an older version of the server application or some other measures. Are these well defined?#2024-05-2707:34Hendrik"user is filling-out a long form" -> for the record, electric makes possible a form architecture where all user interactions are immediately streamed to the server as they type them, where the state can be made durable even before committing to a database
what would be the usecase for this? why would you stream data to the server, if you do not plan to commit it immediately?
Wouldn’t it be much simple from ops perspective to store the data in browser local storage and stream it to server, when you plan to commit the data to db?#2024-05-2707:38xificurCsounds use case dependent. Local storage is by definition local, what if you want the user to start a form on their mobile phone and be able to continue on their notebook? What if you're halfway through the form and you want to delegate the rest to one of your team members?#2024-05-2707:41grounded_sageMost people expect a cross device unified experience these days. But there is definitely reasons for local. #2024-05-2707:45Hendrikok. cross device experience is a fair point. However, the original question was about how to not loose form state on page refresh, which is triggered by a new electric deployment 😉#2024-05-2800:34Dustin GetzConsider for example PKM tools like Notion that can silently lose data if there are connection problems or server problems or the device has low battery and then you lose work. It's my personal opinion that all changes should be "made safe" instantly, and if they are not saved then the input control should be visibly dirty i.e. yellow. The goal being to never lose state ever again. CRUD stuff being an edge case of the more complex Notion use case (which includes datagrids and forms)#2024-05-2703:30eunminFolks, I created an LLM library called https://github.com/constacts/ragtacts. I made an LLM chat playground example using electric. I received a lot of help from this channel, especially regarding file uploads. clojure-spin
https://github.com/constacts/ragtacts/tree/main/examples/playground#2024-05-2709:11grounded_sageI notice that sometimes when I update a .clj file I have to restart electric to get access to the new function I created. Is this just a requirement or is there something I can do to fix this.#2024-05-2709:27xificurCreload-on-save only works on cljc files, shadow doesn't watch clj files. Do you eval the clj file in a clojure REPL?#2024-05-2709:28grounded_sageYes but I haven’t figured out how to connect the REPL to electric since IC update#2024-05-2709:32grounded_sageThis is good to know on how to get the reload on save even if I just do that for development. #2024-05-2710:57xificurCcljc files autoreload
clj files user needs to reload through the REPL. Setup of that is IDE dependent#2024-05-2711:02xificurCe.g. me as a doom emacs user I start the REPL through the setup we used to have in the readme, https://github.com/hyperfiddle/electric/pull/6/files. Then I type (dev/-main) in the REPL to start shadow and the app#2024-05-2711:07grounded_sageI use calva. Used to do a connect to REPL sequence where I chose generic and typed in 9001 if I remember correctly but that stopped working and I was too lazy to check here on how to get going again#2024-05-2717:09Vincentif i name my .clj into .cljc i get free reload hax? can i just wrap everything in one giant reader conditional? lol#2024-05-2718:00grounded_sageYea the reload thing and how to connect to repl both server and client would be nice to have documented somewhere I feel. Even if not on main repo.#2024-05-2718:30xificurCIt would fit in the starter app. PRs welcome#2024-05-2720:51VincentI would write a description if I knew how to do it#2024-05-2803:20grounded_sageYea I also don't know how to connect a repl since the latest release#2024-05-2717:23braai engineerWhat's the fastest JDK for local dev? Oracle Java?#2024-05-2719:33henrikJust learned of https://braid.org, interesting stuff.#2024-05-2800:11VincentI saw that and immediately thought of electric clojure . i think having the "data fresh and accurate since this branch id" will be bandwidth saving and it's hard to believe the internet has existed so long without something like this#2024-05-2911:23Dustin GetzThese document sync technologies are used in applications like Google Docs and other multiplayer editors, the reason we don't see them in other domains is that they are document centric not relational/database centric and thus have scalability issues#2024-05-3006:28henrikAs I understand, they’re not particularly about document sync: “The Braid Protocol allows multiple synchronization algorithms to interoperate over a common network protocol…”
Maybe I’m misunderstanding, but it sounds more like it’s about conveyance than any particular sync strategy.#2024-05-3011:33Vincenthave you played the video game Braid? you can "reverse time" and the music plays in reverse.#2024-05-2823:29nakkayaI am working on a custom type that implements IAtom IDeref IRef IMeta IObj IReference . It acts like an atom but the data is synced to a database. I can use it like the following,
(e/defn Counter []
(let [state (e/server
(e/watch
(my-atom 0)))]
(e/client
(dom/p (dom/text state)))))
which works, any updates to the database automatically updates the UI. What I can't get working is the following form,
(e/defn Counter []
(let [!state (e/server
(my-atom 0))
state (e/watch !state)]
(e/client
(dom/p (dom/text state)))))
I would like to keep a reference to !state so I can, say from a button callback do something like (e/server (swap! my-atom inc)) . Currently latter form results in Unserializable reference transfer.#2024-05-2900:54Dustin Getzlikely the watch is client sited causing an undesired transfer, due to calling Counter on the client#2024-05-2908:14nakkayaIs there a workaround you can suggest? @U09K620SG I would like to avoid having a top level (e/def !state)#2024-05-2908:19xificurC(e/defn Counter []
(e/server
(let [!state (my-atom 0)
state (e/watch !state)]
(e/client
(dom/p (dom/text state))))))#2024-05-2908:23nakkaya@U09FL65DK thanks that did it 👍#2024-05-2908:24xificurCdo you understand why?#2024-05-2908:28nakkayamy understanding is, both atom-like and watch is constructed server side then only watch is transferred back to client, which was also my intent in the first form, third form actually hinted the electric to actually do it?#2024-05-2908:33xificurCatom - reference type, cannot be serialized. Also, it would imply client could reset! a server atom
(e/watch atom) - reactive deref, returns latest value of the atom. Typically the value inside is serializable
Your second snippet constructs a server-side atom and attempts to watch it on the client, which would require transfering the atom itself#2024-05-2917:05VincentHey gents
Has anyone made a dynamic search box with Electric?
I am working on one today.
If I find anything interesting about doing it perhaps I will share.#2024-05-2917:06VincentThinking the "realtime search" where the results display as you type, with some aesthetic modifications#2024-05-2917:12xificurCLike ui4/typeahead?#2024-05-2917:14VincentThat sounds like what I need 😄 what is typeahead ? :;hunts for source code;:#2024-05-2918:43xificurCIt's far from perfect and basically unused. Nevertheless you can try it out, maybe it fits your needs, or at least can give you inspiration.
There are https://github.com/hyperfiddle/electric/blob/a40ee67444d739342a8ec2a8715a84c8295f4b39/test/hyperfiddle/electric_ui4_typeahead_test.cljc documenting its behavior.#2024-05-2919:46cjmurphyIsn't 'realtime search' what electric-tutorial.demo-webview/Webview does?#2024-05-2919:55xificurCmaybe you're right @U0D5RN0S1. The question was underspecified. The typeahead is a picker with search, the webview or system properties demo is a table with a search field on top#2024-05-2920:25nakkayaI am using one, the way I set it up, I have an atom that gets updated from an input field, watch on the atom triggers a db query which updates the table. As the user types table content updates accordingly.#2024-05-2922:22Vincent@U09FL65DK not sure how to incorporate typeahead ... do you have a demo with a working implementation? GPT4o is not a clojurian#2024-05-3006:56xificurCfirst of all, as there is confusion, is the typeahead what you're looking for? Here is an example typeahead from the web#2024-05-3007:02xificurCwe don't have demos of it. You can copy the one from the tests into your app and start playing around with it to see how it works. But in short
• you have to mount it in an e/server
• the args are in order
• a controlled value. This is similar to other ui4 controls
• V! e/fn whose purpose is to transact your chosen value
• Options e/fn that queries for the option list
• OptionLabel e/fn that turns the option into a presentation. E.g. in the picture above Options could return db ids and OptionLabel would turn that id into the country name and flag
• body that runs on the client#2024-05-3011:34Vincent🙏thank you xificurC
given the incidental complexity of wrangling with a fresh component I'll probably wait to implement it until I have an intern to task with it xD#2024-05-3114:49bdbrodienow that I’ll be working on search + LLM demos, I will definitely be interested in this!#2024-05-3003:34JAtkinsIs there a way to prevent the codemirror object from losing focus while typing if it's used as a controlled value? I see it re-mounting after un-commenting (println 'cm-mount) . I would've assumed based on the comment stable-through-cv-changes it wouldn't be re-created all the time?#2024-05-3003:38JAtkinsI see this line being invoked when I change the value in a controlled way
https://github.com/hyperfiddle/electric/blob/a40ee67444d739342a8ec2a8715a84c8295f4b39/src/contrib/electric_codemirror.cljc#L75
This is the line potentially leading me astray
https://github.com/hyperfiddle/electric/blob/a40ee67444d739342a8ec2a8715a84c8295f4b39/src/contrib/electric_codemirror.cljc#L89#2024-05-3009:34Dustin Getzmy quick read of this is that when props change the cm will be rebuilt, you should separate out the controlled value into the explicit parameter and make sure all the other props are unchanging#2024-05-3016:58JAtkinsAfter taking another look it seems that the reductions on >cm-v is rebuilt when controlled-value changes. Which seems right, except that cancels the existing reduction, canceling >cm-v , causing an unmount of codemirror . Does this sound about right?#2024-05-3017:14JAtkinsIf this is right, I think I want to convert (*new* (*m*/*relieve* {} (*m*/*reductions* #(*readf* %2) controlled-value >cm-v))) to something that looks like
(new (m/relieve {}
(m/reductions #(readf %2)
""
(m/latest some-fn-determining-which-is-newest
controlled-value
>cm-v)))#2024-05-3017:19JAtkinshrm, I don't actually need controlled value to be part of this reduction#2024-05-3017:26JAtkinsyeah, I'm dumb. this makes alot more sense. rather unexpected to debug haha.
I'm now using
(new (m/relieve {}
(m/reductions (fn
([a b] (readf b))
([] ::impl-value))
>cm-v)))
#2024-05-3013:01Absolute NegativityHow can I alleviate compiling time? My current project takes up to 30 secs before I see changes in the UI.#2024-05-3013:43Dustin GetzMake sure you are on latest version, in Jan we landed incremental compilation#2024-05-3013:45Dustin GetzFrom that point, optimize by using smaller namespaces, because ClojureScript compilation unit is per file#2024-05-3013:47Absolute NegativityThank you. Will try.#2024-05-3102:56Vincentincremental compilation is super fast#2024-05-3102:57Vincenti have it working in a demo but have yet to update one of my main apps to use it xD so i minimize the file size of the namespace#2024-05-3102:57Vincentby pruning methods / components i don't need, and sometimes moving components out into another file if i want to iterate fast and the app is <huge huge>#2024-05-3114:45bdbrodiewe upgraded a few weeks ago, it was so worth it ⚡#2024-05-3103:19VincentYooo check out how pretty this is
#?(:cljs (defn get-audio-element []
(.getElementById js/document "audioPlayer")))
(e/defn CustomAudioComponent []
(e/client
(dom/div
(dom/props {:class "custom-audio-wrapper"})
;; Custom Audio Player Element
(dom/element "custom-audio-player"
(dom/props {:id "audioPlayer" :src "PL.m4a"}))
;; Play Button
(ui/button
(e/fn []
(.log js/console (get-audio-element))
(.fadeInAndPlay (get-audio-element)))
(dom/text "fade in and Play"))))))
i have a custom audio player and i can write electric code to play it via (.fadeInAndPlay (get-audio-element))
the player is written in javascript, it leverages prototype which is a way to extend native javascript objects to have more functions of your choosing / selection / coding. And so. I highly recommend this approach it is hella easy to implement custom javascript widgets/applets into your bigger electric app and keep distinct javascript components in their own js glory in their own file.#2024-05-3104:54Vincent// Extend HTMLAudioElement prototype to add fadeInAndPlay method
HTMLAudioElement.prototype.fadeInAndPlay = function() {
const audio = this;
audio.volume = 0; // Start from volume 0
audio.play();
const duration = 2222; // 2.222 seconds
const step = 50; // Smaller step for a smoother transition
let elapsed = 0;
const fade = setInterval(() => {
elapsed += step;
audio.volume = 1 - Math.pow(0.1, elapsed / duration);
if (elapsed >= duration) {
audio.volume = 1;
clearInterval(fade);
}
}, step);
};
this is in the index.html page in a <script type='text/javascript'> tag pair, and exposes the element to the clojurescript namespace when you getElementById which makes working in electric super fun even with whack javascript stuff.#2024-05-3104:54VincentI can add "missing functionality" without having to derive new components in cljs. So, that's one benefit of extension / extending prototypes. Just in case you need this tool ever.#2024-05-3109:44Dustin Getzyes!!#2024-05-3109:46Dustin Getzyou can also just use a global JS function and not have to extend the prototype, call the global JS function from cljs and pass it the audio element#2024-05-3109:47Dustin Getzif your JS code gets big, shadow-cljs has ways to require JS modules from cljs :require#2024-05-3113:16Vincentsuper cool#2024-05-3113:16VincentHey I have noticed that on the first cold boot everything is cool but then after messing with the page refreshes I will get "cannot infer type" on my new invocations as a warn#2024-05-3113:32Dustin Getzpaste exact warning please, and also code#2024-06-0123:18Tiago LuchiniI have been using a very simple hiccup to electric compiler on a project for a while and am considering spinning it off as an oss library. Here's how it works both for reference and to measure community interest. Let me know in this thread your thoughts. https://github.com/luchiniatwork/hiccup-for-electric#2024-06-0207:13JAtkinsHow would one debug app-wide instability of the dom? I can see evidence of things changing and I can guess where the instability is created, but I'd like an analytical way to nail down dom flashing.#2024-06-0212:40Dustin GetzHere's a macro that we use defensively in a lot in our CRUD datagrid stuff:
(defmacro ignore-pendings [x]
`(let [!x# (atom (e/snapshot ~x))]
(try (reset! !x# ~x) (catch hyperfiddle.electric.Pending ~'_))
(e/watch !x#)))#2024-06-0212:41Dustin GetzThe problem is Electric v2's pending semantics are poorly defined and leak tons of accidental complexity. We surprised ourselves that in our more complex apps not even we can accurately predict what Electric forms will throw Pending and what the behavior will be#2024-06-0212:42Dustin Getzthe interactions between e/for and client/server transfer inside the body of an e/for is a key place where Pending typically shows up unexpectedly#2024-06-0212:46Dustin GetzAddressing this is the last big thing we're working on in Electric v3, which at it's core is a redesign of e/for and building collection awareness into the network runtime. In v2, e/for is implemented as userland hacks which is a primary source of pain in v2 apps#2024-06-0213:52JAtkinsAs it turns out, I wouldn’t have guessed this! Great info to have though - thanks!#2024-06-0210:53braai engineerCloned my Electric BraaiSim app to another laptop and am getting this error for a dom/link I was not seeing before:
ERROR hyperfiddle.electric: #error {
:cause $Property_expr_4 is not defined
:data {:hyperfiddle.electric.debug/trace [{: :, :hyperfiddle.electric.debug/type :eval, :hyperfiddle.electric.debug/fn {}, :hyperfiddle.electric.debug/args [[#object[HTMLLinkElement [object HTMLLinkElement]] :rel "stylesheet"] <exception>],
Here is the Electric bit:
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(dom/link
(dom/props {:rel "stylesheet" :type "text/css" :href "/braai.css"}))
(braai/Braai.))))
Could it be caused by a JDK version difference? Maybe something that only shows up in dev mode?#2024-06-0212:33Dustin GetzI have not seen this. Please can you confirm the starter app works#2024-06-1517:08braai engineerI can confirm the electric-starter-app works just fine on this late-2019 MBP16, but I am still getting this error in my project after removing the dom/link element.
If I remove the dom/link that causes it, the error just moves to the next component. The error is in https://gist.github.com/theronic/2a8a4bff400aef72719bb5152bbd7392. I left a comment with the Electric component that has the dom/props causing it. The app works but sone of my touch events are broken, presumably due to the missing props.
I thought it was maybe caused by data-attributes in dom/props, but I removed them all and it still throws for simple (dom/props {:draggable true}).
I switched JDKs and double-checked Clojure versions. Issue still occurs on this machine, but not on my M3 Macbook.
Very weird error 😕 Can’t dev on this machine (my other machine’s screen is broken).
Any ideas what I can check next short of wiping the machine?#2024-06-0215:47braai engineerWhen deploying a new version of my app to http://Fly.io (built on Electric Starter App), I get the following error:
5.600 Checking out: at b32ac98df7d7ec87f225d47354671be172ffa87e
6.661 Error building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.12.0-alpha5
6.662 org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:jar:1.12.0-alpha5
Could be a Maven access issue on Fly, or maybe the alpha artifact is not available? Trying with newer alpha Clojure now.#2024-06-0215:49braai engineerHmm, same error when using org.clojure/clojure {:mvn/version "1.12.0-alpha12"}#2024-06-1513:34braai engineerThis resolved after a few tries. http://Fly.io issue.#2024-06-0518:17VincentHi everyone, gm )#2024-06-0518:25VincentI re-used a login function from another application, changed the name of a variable, and had to hunt it for a while xD#2024-06-0518:26VincentElectric ⚡️ has made a massively multiplayer project possible with what i reckon is probably the smallest possible software development / machine coaxing team. excited for upcoming release#2024-06-0606:57Awab Khanhello i am new to clojure and i just discovered hyperfiddle/electric now i am a bit confused about electric do i need to know clojure to program in electric or do i just have to learn the syntax by brute forcing it since there is no pdf documentation and i am a bit confused about the syntax used in the examples.#2024-06-0607:09teodorlufirst asked in #beginners, perhaps a better fit here. https://clojurians.slack.com/archives/C053AK3F9/p1717656693523739#2024-06-0617:43VincentHmm. You can learn a lot from the examples. https://electric-examples-app.fly.dev. If there is something unclear, ask here#2024-06-0617:44Vincent@U3X7174KS thanks for reposting here, teo#2024-06-0617:59Vincent@U076U9EU09Z in general, find an example closest to what you need, duplicate the repo from github exactly, and use that as a starting point for your project.
(e/server ...) colors or tints code for clojure compilation [backend server]
(e/client ...) colors or tints the code for browser/clientside compilation via clojurescript which has full access to the javascsript context and the html DOM.
you can: mix and match
(e/for-by :track/id [track (e/server (e/offload #(db/get-tracks)))]
(e/client
(dom/div (dom/text "track id is " id))))
will print out all the track ids in divs, for example. e/server has the result, e/for-by iterates "electrically" over the serverside result and spits out clientside divs.#2024-06-0709:42Awab KhanThanks for the help i really appreciate that.#2024-06-0713:19VincentSure thing, anything to help you get started and with traction.
Essentially you have 3 layers now, clojure, clojureScript (equals javascript), and Electric.
You need to "lift" atoms into the electric layer via
#?(:cljs (def !client-atom (atom "starting value")))
(e/defn client-atom-live (e/client (e/watch !client-atom))
#?(:clj (def !server-atom (atom "start val")))
(e/defn server-atom-live (e/server (e/watch !server-atom)))
Notice the naming convention, if it lives in the client layer or server layer (i.e. it is /they are pure cljs or pure clj)
then it gets an exclamation point#2024-06-0713:19VincentAnd the electric layer is the naked version or shoutless version i)#2024-06-0618:41nakkayaI've just upgraded my app to match the latest electric-starter-app . Previously I had,
(let [jetty-options (merge {:port 8080
:join? false
:websockets {"/" (fn [ring-req]
(adapter/electric-ws-adapter
(partial adapter/electric-ws-message-handler
(-> ring-req
(session/session-request
{:store cookie-storage
:cookie-name "ring-session"})))))}}
config)
server (ring/run-jetty ring-handler jetty-options)]
;;;
)
then I was able to access session data using (e/server (get-in e/*http-request* [:session :identity :clinic-id])) after my changes I no longer have :session data in e/http-request . How should I access session data when using the new electric-websocket-middleware ?#2024-06-0619:31cjmurphyUsing [a modified] electric-starter-app I can get a session-id like this:
(e/server (get-in e/http-request [:headers "sec-websocket-key"]))
#2024-06-0620:08nakkayaDuring login I stuff some extra things in the cookie those are the things I am trying to access after electric starts.#2024-06-0620:33cjmurphyI forgot to mention I'm doing this binding as something extra to the electric-starter-app :
(binding [e/http-request ring-request]
Doing that will be a prerequisite for any accessing of e/http-request.#2024-06-0711:10nakkayaHi, I am having some issues migrating from v2-alpha-469-gb6d9865c — 2023 Nov 22 to v2-alpha-540-ga4699532 — 2024 Jan 10 . On the old version I had the following working for routing within the app,
(ns browser
(:require [clojure.string :as str]
[missionary.core :as m]
[hyperfiddle.electric :as e]
[goog.events])
(:import [goog.history Html5History EventType]))
(deftype QueryAgnosticTransformer []
Object
(createUrl [this token path-prefix location]
(str path-prefix token))
(retrieveToken [this path-prefix location]
(let [path (.-href location)]
(.substr path (count path-prefix)))))
(defn new-goog-history []
(doto (Html5History. js/window (QueryAgnosticTransformer.))
(.setPathPrefix (str js/window.location.protocol
"//"
js/window.location.host
"/app"))
(.setUseFragment false)
(.setEnabled true)))
(defn path> [^js h]
(m/observe (fn mount [!]
(let [k (goog.events/listen
h EventType.NAVIGATE
(fn [^js e] (! (.-token e))))]
(! (.getToken h))
(fn unmount []
(goog.events/unlistenByKey k))))))
(e/def !history (new-goog-history))
(e/def path (new (m/relieve {} (path> !history))))
(defn navigate! [^js !history href]
(.setToken !history href))
then on the UI I do,
(e/defn Page []
(e/client
(let [match (r/match-by-path router browser/path)]
(if-let [view (get-in match [:data :name])]
(let [params (:path-params match)]
(condp = view
::page-a (PageA.)
::page-b (PageB. params)
(PageA.)))
(PageA.)))))
when ever I want to navigate to a new page I do,
(browser/navigate! browser/!history (str "/page-b/" id))
but on v2-alpha-540-ga4699532 URL updates but Page does not redraw. The only change I did during the port was to rename the file from cljs to cljc and color it to get it to compile,
#?(:cljs
(deftype QueryAgnosticTransformer []
Object
(createUrl [this token path-prefix location]
(str path-prefix token))
(retrieveToken [this path-prefix location]
(let [path (.-href location)]
(.substr path (count path-prefix))))))
#?(:cljs
(defn new-goog-history []
(doto (Html5History. js/window (QueryAgnosticTransformer.))
(.setPathPrefix (str js/window.location.protocol
"//"
js/window.location.host
"/app"))
(.setUseFragment false)
(.setEnabled true))))
#?(:cljs
(defn path> [^js h]
(m/observe (fn mount [!]
(let [k (goog.events/listen
h EventType.NAVIGATE
(fn [^js e] (! (.-token e))))]
(! (.getToken h))
(fn unmount []
(goog.events/unlistenByKey k)))))))
(e/def !history (e/client (new-goog-history)))
(e/def path (new (m/relieve {} (path> !history))))
#?(:cljs
(defn navigate! [^js !history href]
(.setToken !history href)))#2024-06-0721:19nakkayaIn case anyone else stumbles on this I was able to get it work by binding !history and path on boot,
(binding [browser/!history (browser/new-goog-history)]
(binding [browser/path (new (m/relieve {} (browser/path> browser/!history)))]
(App.)))#2024-06-1109:05Vincenti assume there's some logic to nesting the bindings?#2024-06-1109:07nakkaya@U055PQH9R4M btw forgot to mention it in your other message but this what you are looking for, this will allow you to add bookmarks to views with in the app.#2024-06-1109:10nakkayaI may be wrong on this one but from another thread in this slack, binding happens in parallel so one binding wont see the other hence the nesting. @U09K620SG#2024-06-1109:23xificurCyes, bindings happen in parallel, same as in clojure(script). Since browser/path depends on browser/!history one needs 2 binding calls#2024-06-1116:18Vincentcool#2024-06-1116:18Vincentthanks i am glad you posted this, it confuses me a little bit, i think you have a browser/ namespace?#2024-06-1118:11VincentHmm. I think you have a browser.cljc file, would you mind sharing the require statement#2024-06-1118:16nakkaya@U055PQH9R4M yes top snippet lives in browser.cljcfile. ns decleration looks like this,
(ns browser
(:require
#?(:cljs [goog.events])
[hyperfiddle.electric :as e]
[missionary.core :as m])
(:import
#?(:cljs [goog.history Html5History EventType])))
#2024-06-1118:17VincentOhh I see okay I was so confused lol#2024-06-1118:17VincentThank you @U5H4U2HEH#2024-06-1118:43Vincent@U5H4U2HEH in your binding statement do you need to tint it via e/client somewhere?#2024-06-1118:50Vincentit might be because i'm using the previous version, bit of a minefield 😅 appreciate your patience#2024-06-1119:27nakkayaI am on version v2-alpha-540-ga4699532 — 2024 Jan 10 this is how I bootstrap everything,
(e/defn Main [ring-request]
(e/server
(binding [e/http-request (session/session-request
ring-request
{:store (cookie-store {:key db/cookie-secret})
:cookie-name "ring-session"})]
(e/client
(binding [dom/node js/document.body]
(binding [browser/!history (browser/new-goog-history)]
(binding [browser/path (new (m/relieve {} (browser/path> browser/!history)))]
(App.))))))))
#2024-06-1204:07VincentWow very elegant#2024-06-0711:59Awab Khanso i installed lein and made a new app by using
> lein new app main
then i added the following dependencies to project.clj file:
(defproject main "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url ""
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url ""}
:dependencies [[org.clojure/clojure "1.11.1"]
[com.hyperfiddle/electric "v2-alpha-540-ga4699532"]
[ring/ring-core "1.12.1"]
[ring/ring-jetty-adapter "1.12.1"]]
:main ^:skip-aot main.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all
:jvm-opts ["-Dclojure.compiler.direct-linking=true"]}})
now i copied the code from the hyperfiddle live tutorial for the counter app to see if i could run it. made the necessary changes then ran the command, lein run the code ran fine because i didn't get any errors. but to show it on localhost i need to run the jetty server but i couldn't figure out how to set it up there was a jetty file on the electric-starter-app but i couldn't understand it and how to run it with lein. do i copy the whole file to my main namespace where core.clj is and just run lein. I just want to start jetty with minimal config to see that i can run electric. I hope i am making sense. any help is appreciated.
now i am getting these errors my core.clj looks like this:
(ns main.core
(:gen-class)
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
(defn -main []
(e/defn Counter []
(e/client
(let [!state (atom 0)]
(dom/p (dom/text (e/watch !state)))
(ui/button (e/fn [] (swap! !state inc))
(dom/text "Count"))))))
errors:
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
i apologize in advance if i have missed something clear and coming of as ignorant.#2024-06-0712:30telekidI recommend you start by cloning the starter app and modifying it#2024-06-0712:31telekidI saw in your earlier thread that you're new to Clojure. How comfortable are you with the basics of the language? Electric is amazing, but it is very lightly documented and debugging will be challenging without some experience in Clojure.#2024-06-0712:34iperdomohttps://clojure.org/guides/install_clojure and https://github.com/hyperfiddle/electric-starter-app#2024-06-0712:45Dustin GetzPlease clone and modify the starter app, the readme contains instructions on how to run it#2024-06-0712:53Awab KhanThanks i'll start with electric-starter-app first.#2024-06-0721:19Nikolas PafitisIs there anyone willing to pair with me, that has tried integrating electric with polylith in the past and managed to set up the development environment and then also be able to build the app in "prod"?#2024-06-0723:02Piotr RoterskiI have such setup more or less figured out. What problems are you running into? I’m up for pairing on this, feel free to dm me#2024-06-0814:48Dustin Getzsee also https://github.com/jeans11/demo-rama-electric which i found by searching the channel history#2024-06-0819:31Nikolas PafitisI saw it before but it's not exactly the direction i want to go. Also i think it uses an older version with an older api.#2024-06-1006:39HendrikI have a electric polylith setup. Unfortunately, I have no time for pairing at the moment. My setup looks roughly like this:
• a base that contains the starter app
• move shadow-cljs.edn and package.json to the workspace root
• create a sublayer under resources that like this my_base/resources/my_base . this will help you to keep your js, css … clean
• create a build-target for my_base in shadow-cljs.edn
• have all dependencies in package.json for the dev project
• create a polylith project
• create a package.json there, which only contains npm packages for this project
• move build.clj into the project dir and adjust paths to my_base
• create a shadow-cljs.edn for the project, which contains settings for a prod build and has relative path to my_base
For docker:
• create a dockerfile at workspace root with this content
• copy the base, components and project (keep the relative structure
• set workspace dir to projects/my-project
• set Version and run the uberjar build target
• now you have a prod uberjar, continue from here as you need
#2024-06-1218:39Nikolas PafitisHi @U023LKF2PQV thanks for the response. I did something similar to what you said, only I'm trying an example with 2 projects. Right now package.json is on root level, but it would be in project directory as you suggested. I'm having troubles properly serving the electric app. Could you take a look at it? https://github.com/conjurernix/streamcraft/#2024-06-1218:45Nikolas Pafitis#2024-06-0819:33Nikolas PafitisAnyone knows what might be causing this? All js resources load fine by the browser, the only thing that's getting a 500 is favicon.#2024-06-0822:05Nikolas Pafitis@U09K620SG maybe?#2024-06-0823:13Nikolas PafitisI figured out my mistake, It turned out i wasn't using boot-server although i was using boot-client. After fixing that and consolidating client-version and user-version for wrap-reject-stale-client middleware I manage to make some progress. Now I'm stuck with this:#2024-06-1012:01nakkayaJust as an FYI following snippet,
(dom/span
(dom/props {:class "badge badge-info"})
(let [size (-> (get-in state [:events :storage])
(/ 1024)
(/ 1024))
size (cond (zero? size) "0 MB"
(< size 1024) (str ( "%.2f" size) " MB")
:else (str (gstring/format "%.2f" (/ size 1024)) " GB"))]
(dom/text size)))
worked fine on v2-alpha-469-gb6d9865c for both prod and dev builds, however on v2-alpha-540-ga4699532 it works in dev build but fails on prod build [1]. Replacing gstring/format with toFixed solves the issue. @dustingetz
[1] https://gist.github.com/nakkaya/a4d441f93b30d6136f8a6343cc1d327d#2024-06-1104:23VincentAny efforts towards an e/navigation ?#2024-06-1104:24VincentNeed: SPA (single page application) requires persistent URLs for sanity and browsing and bookmarking and SEO, maybe there is a smart way to navigate views once the electric app hs loaded.#2024-06-1109:10nakkayaSee [1].
[1] https://clojurians.slack.com/archives/C7Q9GSHFV/p1717758606163139#2024-06-1109:31Dustin Getzwe are working on some “batteries included” features like this but nothing good enough to release widely yet. electric fiddle has a powerful application router but it’s complicated #2024-06-1111:14henrikWildcard your routes to index in the backend router and use whichever frontend router you want. We use Reitit, and it works well.#2024-06-1119:06Vincentallergic to dependencies xD#2024-06-1109:02Vincentis there a way to catch the "connected" event#2024-06-1112:46Dustin Getzhttps://xyproblem.info/ please tell us what you are trying to do#2024-06-1113:10Dustin Getzare there any clojure frameworks (fulcro reframe etc) that have a good virtual scroll solution that can scroll over 1000s of records?#2024-06-1210:20danielnealI don’t think there’s anything out of the box.
I think probably in reframe / etc only using a library (react-virtualized?).
Also I remember hoplon had for-tpl which might be used for this sort of thing (https://github.com/hoplon/hoplon/wiki/Dynamic-DOM-manipulation-aka-Template-Macros)#2024-06-1210:21danielnealBut nothing actually built as a reusable component I think#2024-06-1211:38Dustin Getzso do clojurescript apps just not have virtual scroll?#2024-06-1211:41danielnealThe last time I needed something like this was in react native, and I used some virtualized list from js#2024-06-1211:41Dustin Getzpopulated how, rest api with pagination offsets?#2024-06-1211:42danielnealThe database (it was a product catalogue) was pretty small and embedded in the app and could fit in memory - the virtualization in my case was only needed for smooth scrolling / performance as each list item was a card with images and icons and reusing the cells improved performance. All I remember was that it was fiddly to get right and I tried a bunch of different libraries before getting something that worked.
I think in the end they implemented something for this purpose in react native itself (https://reactnative.dev/docs/flatlist)#2024-06-1211:42danielnealI’m probably the wrong person to have answered this haha#2024-06-1507:09Mario TrostOnly used it once directly in JS, but I would probably use `tanstack/virtual
https://tanstack.com/virtual/v3`
#2024-06-1507:10Mario TrostIn combination with tanstack/query and tanstack/table#2024-06-1114:56denikfor Clojurians in NYC, I’ll be giving a talk about electric: https://www.meetup.com/clojure-nyc/events/301525732/ in a week#2024-06-1806:14mattiaswould love to see a recording as well#2024-06-2218:49mattiasReally cool project! I've https://github.com/subspace-ai/subspace (but not really developing) a roam-like tool with a builtin repl and Electric rendering as sth you would use for internal tooling and managing personal/project wikis. There the basic unit of UI would be a node (where nodes can be nested under other nodes and collapsed, it's easy to rearrange these with just keyboard, etc), whereas you went with the spreadsheet approach that has other useful characteristics. Still tesserae has many of the things I imagine for such a roam-like tol: code eval + Electric rendering, schedules, service workers, referencing other nodes (in your case cells). It would be pretty useful to be able to combine the two paradigms IMO - a PKM where certain nodes are spreadsheets, and spreadsheet cells can reference nodes defined elsewhere in the graph.#2024-06-2416:43denik@U023LKF2PQV there’s a commit in the readme for which it definitely works!#2024-06-1210:00nakkayaAfter upgrading to v2-alpha-540-ga4699532 one of my forms started generating the following exception. [1] It happens intermittently restarting the app fixes it for a while, form itself works without any issues.
[1] https://gist.github.com/nakkaya/982bcc18311f58d32247382ec98e9cd4#2024-06-1222:03Nikolas PafitisI have managed to solve my previous issues posted above. What I found i was doing wrong was this:
Instead of using
(def entrypoint (e/boot-client {} App nil))
and referencing the App that was e/defn 'ed directly, I was trying to do something of the sort of:
(defn entrypoint [app] (e/boot-client {} app nil))
which the e/boot-client macro didn't like, and the console error did not help.#2024-06-1223:36Dustin Getzsorry about the debugging pain#2024-06-1417:51Nikolas PafitisNo worries, it's to be expected 🙂#2024-06-1716:43denikGetting warnings from shadow-cljs when compiling this file https://github.com/lumberdev/tesserae/blob/master/src/tesserae/ui/electric_util.cljc#L48
WARNING: Use of undeclared Var tesserae.ui.electric-util/
seem to be false positives. everything resolves properly and app works flawlessly. how can these be turned off?conn and db. To address this, I created a state atom and reset the state with data from the database. Now, when the delete button is clicked, two functions are triggered: one to remove the entity from the state and another to delete it from the database. This solution works for now, but I'm unsure if it's the best approach.
I might not have explained this clearly, so feel free to check the project https://github.com/collabryio/datomic-refresh. I would appreciate any suggestions or feedback. Thanks!#2024-06-2019:43Dustin Getzyou need to reactively rebuild the db val when transactions happen via the datomic tx listener api, use search to find past discussions#2024-06-2020:02Hüseyin YavaşI'll do it. Thanks!#2024-06-2020:45AHMET OĞUZHAN ENGİNHello everyone,
I'm working on a project using ClojureScript and Hyperfiddle, and I'm encountering some issues with login and logout functionality. When a user logs in, a token is created and added to localStorage. I access this token using the get-tokenfunction, and in the is-logged-in function, I check the existence of this token to set the user's state. However, I'm facing two issues:
1. Login Text Not Changing to Logout:
◦ When a user logs in, the "Login" text should automatically change to "Logout," but this change doesn't happen unless I manually refresh the page.
2. Logout Button Not Working:
◦ When the logout button is clicked, it should remove the token from localStorage and refresh the page, but the onClick event doesn't seem to be working.
Below is the relevant code:
(ns pages.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.router :as router]
[hyperfiddle.history2 :as h]
#?(:clj [datomic.client.api :as dt])
#?(:cljs [reagent.core :as r])
#?(:cljs [reagent.dom :as rdom])
#?(:cljs ["react" :as react])
#?(:cljs ["react-dom/client" :as ReactDom])
pages.login))
(e/def conn)
(e/def db)
;; User Token
(e/defn get-token []
(e/client
(. js/localStorage getItem "sessionToken")))
(e/defn is-logged-in []
(let [!state-button (atom false)]
(if (nil? (get-token.))
(do (reset! !state-button false)
(println @!state-button)
@!state-button)
(do (reset! !state-button true)
(println @!state-button)
@!state-button))))
(e/defn logout []
(e/client
(. js/localStorage removeItem "sessionToken")
(router/Navigate!. ['.. [::login]])))
(e/defn bootstrap-navbar []
(e/client
(dom/link (dom/props {:rel :stylesheet :href "/assets/css/banner.css"}))
(dom/nav (dom/props {:class "navbar navbar-expand-lg navbar-dark bg-dark"})
(dom/div (dom/props {:class "container-fluid"})
(router/link ['.. [::home]] (dom/a (dom/props {:class "navbar-brand"})
(dom/img (dom/props {:src "/assets/images/logo.png", :alt "Logo", :width "140", :height "90", :class "d-inline-block align-text-top"}))))
(dom/button (dom/props {:class "navbar-toggler", :type "button", :data-bs-toggle "collapse", :data-bs-target "#navbarSupportedContent", :aria-controls "navbarSupportedContent", :aria-expanded "false", :aria-label "Toggle navigation"})
(dom/span (dom/props {:class "navbar-toggler-icon"})))
(dom/div (dom/props {:class "collapse navbar-collapse ", :id "navbarSupportedContent"})
(dom/ul (dom/props {:class "navbar-nav me-auto mb-2 mb-lg-0 "})
(dom/li (dom/props {:class "nav-item"})
(dom/div (dom/props {:class "nav-link active text-light"})
(router/link ['.. [::home]] (dom/a (dom/text "Home"))))))
(if (is-logged-in.)
(dom/div
(dom/button (dom/props {:type "button"
:class "btn bg-dark border border-light text-light"
:onClick (logout.)})
(router/link ['.. [::login]]
(dom/span {:style {:color "white" :text-decoration nil}}
(dom/text "Logout")))))
(dom/div
(dom/button (dom/props {:type "button"
:class "btn bg-dark border border-light text-light"})
(router/link ['.. [::login]] (dom/a (dom/props {:style {:color "white" :text-decoration nil :text-decoration-line nil}})(dom/text "Login")))))))))))
(e/defn HomePage []
(e/server
(binding [conn @(requiring-resolve 'dev/datomic-conn)]
(binding [db (dt/db conn)]
(e/client
(dom/text "Home Page"))))))
(e/defn Main [ring-request]
(e/server
(binding [e/http-request ring-request]
(e/client
(binding [dom/node js/document.body]
(bootstrap-navbar.)
(router/router (router/HTML5-History.)
(let [[page x :as route] (ffirst router/route)]
(if-not page
(router/Navigate!. [[::home]])
(router/focus [route]
(case page
::home (e/server (HomePage.))
::login (e/server (pages.login/RenderLoginPage.))
(e/client (dom/text "no matching route: " (pr-str page)))))))))))))#2024-06-2107:14xificurC• the effect (. js/localStorage getItem "sessionToken") runs only once, since electric only re-runs it if any of its arguments change
• (dom/props {:onClick ..}) is not a thing. Use dom/on or dom/on!
Have you gone through the https://electric.hyperfiddle.net/?#2024-06-2201:56grounded_sageIt’s not fully clear to me what happens with clients connected when a new deploy happens. Does it just lose the websocket connection and they have to refresh? How is automatic reconnection/maintaining auth handled for a production application?#2024-06-2202:50Dustin Getzclients whose connection was dropped will auto refresh, auth is on you, a cookie should be fine#2024-06-2220:04simonkatzCan I use dom/input for a multi-line input? Or, failing that, is there something else that supports that?
(When starting to look into this, it looks like it’s removing any newlines when I use (set! (.-value dom/node) …).)#2024-06-2220:12Dustin Getztry textarea#2024-06-2220:34simonkatzGreat — thanks!#2024-06-2418:12Vincenthttps://www.flow-storm.org let's electrify this#2024-06-2418:54JAtkinswhatcha mean? fwiw it's java-fx app so somewhat different than current use cases#2024-06-2420:22Hendrikfrom my experiments with electric threejs bindings I would guess that writing electric bindings for java-fx should be pretty simple. I assume that no client/server is needed for this app. In this case you could maybe leverage e/run to make the program run without a client-connection :thinking_face:#2024-06-2609:35euccastroThe electric starter app has a different sha of the com.hyperfiddle/electric dependency than the electric fiddle. I imagine the latter is more recent, and the last-known-good version? Would it work to just check out the starter app and bump the sha to match the one in the electric fiddle?#2024-06-2610:43nakkayaFor keeping track of last good version I personally follow the change log [1]. Whatever is listed last I assume to be last known good version.
[1] https://github.com/hyperfiddle/electric/blob/master/CHANGELOG.md#2024-06-2611:28Dustin Getznakkaya's approach is safest long term; right now master is stable and you can upgrade to latest sha from master#2024-06-2612:22Dustin Getzwe should probably release the current master sha, we're just overwhelmed trying to ship electric v3, so v2 has been in emergency fixes only mode#2024-06-2708:48euccastroThanks! I've upgraded to master and everything works fine so far#2024-06-2612:25Dustin GetzUpdate on Electric v3 - we have a working dom layer, we're upgrading all our tutorial apps and in "fix clojure typos and crashes" mode. No solid date for public beta yet but we are making rapid progress!#2024-06-2614:59danielnealMassive congrats!! Excited to see it 🙂#2024-06-2711:12henrikFantastic. Are you happy with it thus far? Exhibiting the properties you were looking for etc.#2024-06-2618:26denik@hendrik.levering asked me to repost my talk on a electric clojure spreadsheet in this channel so here it is: https://youtu.be/nEt06LLQaBY?si=Jx2wTmDfrSNZM645#2024-06-2710:19mmerReally interesting. How long did it take you to get your head round the electric paradigm.#2024-06-2715:36denikelectric is dead simple. way simpler than any alternative of cross environment (e.g. client+server) architectures. so not long at all! it removed so much complexity and lines of code that really only a few bugs (since it’s alpha software) slowed down dev a little#2024-06-2708:51euccastroIs hyperfiddle.electric-ui4/tag-picker supposed to skip calls to Options if you type too fast for these calls to return in time?
If so, that doesn't seem to work for me. I get one Options callback for every keystroke, so the options popup keeps updating several seconds after I stop typing#2024-06-2808:41braai engineerhaven't tried to replicate, but do your option DOM elements have a unique key if in e/for-by?#2024-06-2811:49Dustin Getzthere are a couple things this could be - the ui4 namespace is not in production anywhere, so the short answer is we don’t understand how it behaves in real world and it’s certainly wrong#2024-06-2811:50Dustin Getzall our work on production crud forms is happening on v3#2024-07-0215:11euccastroThanks! After trying a few things, I vendored it and slapped a throttle (I know, I know) to it and that is satisfactory for now. If I have similar problems once v3 is out I'll look deeper#2024-06-2709:07braai engineerScaling question for Electric Clojure on http://Fly.io: is there anything special I need to configure if I want http://Fly.io to scale up in the event of an extreme traffic spike? Currently only using Electric for frontend rendering, but obviously it is served by the backend.#2024-06-2716:42braai engineerDoes the electric-starter-app support health checks on http://Fly.io? I can't get either TCP or HTTP health checks to work, e.g. in fly.toml:
[[services.tcp_checks]]
interval = '15s'
timeout = '2s'
grace_period = '1s'
[[services.http_checks]]
interval = 10000
grace_period = "5s"
method = "get"
path = "/"
protocol = "http"
restart_limit = 0
timeout = 2000
tls_skip_verify = false
Problem is that GitHub Actions runs more a looong time waiting for fly health checks to pass, and have to cancel the workflows manually.#2024-06-2808:05weithis seems to work for me, not sure if it's correct:
[[http_service.checks]]
grace_period = "1m"#2024-06-2808:39braai engineerthx @U066TMAKS, is your server_jetty.clj file identical to electric-starter-app?#2024-06-2808:40braai engineer@U066TMAKS do your TCP checks work? If yes, what are your parameters?#2024-06-2808:45braai engineerdoesn't work :/#2024-06-2810:02braai engineereventually GH actions times out after about ~8 minutes, but app gets deployed.#2024-06-2810:14braai engineeralso tried with "https" health check, in case there was an HTTP -> HTTPS redirect (which there is), but no dice.#2024-07-2213:26braai engineerthx @U066TMAKS, I got the HTTP health working with grace_period = "1m", but still struggling with the TCP health check.
Is the TCP health check supposed to work?#2024-06-2809:55braai engineerWhat is the current state of offline-first, i.e. if connectivity is lost and client reconnects to server? Last I checked views would go away. Some potential adopters are asking about it.#2024-06-2810:39henrikIt’s online-only. If you don’t have a server connection, you’re effectively working with a graph that includes assumptions of a resource that isn’t available.#2024-06-2811:45Dustin Getzviews will unmount on disconnect for a while longer. we prototyped a fix for this a year ago which was blocked on technical debt, i think v3 unblocks it, we will revisit after v3 ships#2024-07-0214:00Vincentelectric-starter-app is Incremental Compilation ready? (just confirming for sanity sake)#2024-07-0214:12VincentWho among us is a bad enough dude to help me trace this StackOverflow (actual stack frame overflow)#2024-07-0214:39xificurCyes, the starter app runs on a version of electric containing incremental compilation#2024-07-0506:48Vincentthanks#2024-07-0402:45quan xingUsing Electric, do I need to learn other libraries such as Regent, etc.?#2024-07-0404:02Jordan CalderwoodOnly if you need/want to use React.#2024-07-0405:47quan xingSuch as opening a dialog box and then getting the data in the dialog box from the api. In this kind of scenario, if using Electric, how would it be implemented roughly?#2024-07-0407:03mattiasWhen you don't need the UI state persisted in the URL (i.e. after refreshing the page the dialog resets to being hidden) you would have sth like the following.
(e/defn Dialog [!dialog-url]
(e/client
(let [dialog-url (e/watch !dialog-url)]
(when dialog-url
(dom/div
(dom/props {:class "dialog"})
(dom/h2
(dom/text "Dialog ")
(dom/a (dom/text "[x]")
(dom/on "click" (e/fn [_] (reset! !dialog-url nil)))))
(dom/div (dom/text (e/server (-> dialog-url
http/get
:body
json/read-str
:content)))))))))
(e/defn Main []
(e/client
(let [!dialog-url (atom nil)]
(dom/div (dom/text "Main content"))
(dom/a (dom/text "Open dialog")
(dom/on "click" (e/fn [_] (reset! !dialog-url ""))))
(Dialog. !dialog-url))))
We would mount the Main "component" (Electric function) which also instantiates the Dialog function. Dialog initially renders nothing because dialog-url is nil. Clicking on the "Open dialog" link sets the !dialog-url atom state to dialog-url). This in turn causes dialog-url to be non-nil inside the Dialog function, which causes the HTML nested in it to it to be added to the DOM and visible to the user. In parallel, the stuff inside the (e/server) form gets executed: a HTTP GET request to loading.. UI element while the backend http request is executing, or simply wait until the GET is finished before showing the dialog. When you click on the [x]link, the !dialog-url is set to nil again and the DOM elements related to the dialog get removed.
If you would want the UI state persisted in the URL, then dom/on "click" functions would modify the URL instead of an atom, and the frontend components would derive their UI state-related vars (such as dialog-url ) from the URL. Any change to the URL would cause the corresponding vars to change, which would cause different parts of the Electric DAG to run and changed UI to be shown to the user. There is a router implementation in Electric v2 for this kind of stuff (`hyperfiddle.router`) but AFAIK it's not yet documented. It seems pretty powerful but not quite newby-friendly atm. The earlier solution was to add a var that watches the URL and parses the result as a map, so you can pluck out values from this hashmap and pass them down to different components. Not sure what the recommended way of doing this in the upcoming Electric v3 is.#2024-07-0408:21quan xingthanks.#2024-07-0408:25quan xingAre there any open-source projects developed using Electric that I can study?#2024-07-0408:33mattiasIMO the coolest project is this spreadsheet built with electric [1, 2], but it's pretty advanced so I recommend looking at the [3] electric starter app first
1. https://www.youtube.com/watch?v=nEt06LLQaBY
2. https://github.com/lumberdev/tesserae
3. https://github.com/hyperfiddle/electric-starter-app#2024-07-0410:04quan xingthanks#2024-07-0506:48VincentWith uberjar deploy, is there something special to do to ensure database works?#2024-07-0520:19nakkayaI did not had to do anything special, what is the issue?#2024-07-0521:42VincentI think it's just my unfamiliarity with mount and read-config that is causing my current probs#2024-07-0522:31nakkayaIf we are talking about backend db, I just have ns with all my db calls I just include it from electric files and just call with e/server no special treatment.#2024-07-0723:10VincentThank you, my difficulty was summarized by deploying when exhausted and looking at it again with clear eyes. I think I forgot to copy in an entire namespace and was surprised when things almost worked but did not Haha. Thanks always for your indefatigable help#2024-07-0614:33braai engineerMay have found a click bug on mobile where double-tapping a button with side-effects causes an infinite click loop (sometimes double-tap zooms). Could be related to existing loop “click” issues, but mutations are in a clj function and wrapped in e/snapshot. Will show code.
(Tapping “Finish Turn” adds 3 points to the current team and cycles teams.)#2024-07-0614:35braai engineer(dom/button
(dom/on "click" (e/fn []
(js/console.log "next card!")
(e/server
(e/snapshot
(e/discard
(finish-turn! conn game-id player-id))))))
(dom/text "Finish Turn"))
...where finish-turn! is clj fn that takes conn and calls datascript.core/transact!.#2024-07-0618:28xificurCTry putting the snapshot on the arguments that depend on db, probably game-id and player-id.
(finish-turn! conn (e/snapshot game-id) (e/snapshot player-id)))
#2024-07-0618:46braai engineerCould it be because the click handler had no argument, (e/fn [] ...) instead of (e/fn [e] ...)?#2024-07-0618:52braai engineerOK, fixed. Did both. Not sure if it was the snapshot on player-id (which changes between turns), or the event e/fn argument.#2024-07-0619:27xificurCIf you remove one of the changes you'll find out ;)#2024-07-0619:29xificurCMost likely player-id was cycling between v -> Pending -> v ..., busting the work skipping#2024-07-0708:45braai engineer@U09K620SG would be cool if the compiler could warn me of potential loops by printing out a warning, e.g. "Your DOM e/fn at Line 123 captures local state. Use e/snapshot to prevent loops." Hard to be analyze, tho.#2024-07-0709:03braai engineerFixed: I had my e/defn wrapped in #?(:cljs ...) from previous cljs-fn. @dustingetz compiler should probably be able to pick up on this?
Hmm no, still broken. I think it's because I'm calling OnLocation. from a cljs function. Hmm, do I need e/run here?
~Trying to use an Electric Fn for client-side geolocation with signal Fn for position or error, but am getting ~
#?(:cljs
(e/defn Geolocate
"calls OnGeolocation with {:keys [lat lng]} on success, or OnError with error description."
[OnGeolocation OnError]
(e/client
(js/console.debug "Attempt geo")
(if-let [geo (.-geolocation js/navigator)]
(try
(.getCurrentPosition geo
(fn [pos]
(js/console.log "got geo:" pos)
(if-let [coords (.-coords pos)]
(let [pos {:lat (.-latitude coords)
:lng (.-longitude coords)}]
(js/console.log "Position: " (clj->js pos))
(OnGeolocation. pos))
(OnError. "coords missing from geolocation response")))
(fn [error]
(js/console.error "geo error:" error)
(OnError. (pr-str error)))
(clj->js {:timeout 3000
:maximumAge (* 24 60 60 1000)
:enableHighAccuracy false}))
(catch js/Error error
(OnError. (pr-str error))))
(OnError. "Geolocation API not available.")))))
What am I doing wrong here?
Use like so:
(dom/on "click" (e/fn [e]
(Geolocate.
(e/fn [pos] (reset! !geo pos))
(e/fn [error] (reset! !geo-error error)))))
Surprised by this because I know I can do this in an e/fn:
(r/Navigate!. [[::route]])
#2024-07-0709:24braai engineerI suspect I need to use e/run here but not sure how.#2024-07-0712:01Dustin Getzdon't use e/run - you should only have one electric entrypoint in your program#2024-07-0712:01Dustin Getz> I think it's because I'm calling OnLocation. from a cljs function
yes, that is illegal#2024-07-0710:04braai engineerManaged to solve my geolocation issues with a promise and await-promise interop:
#?(:cljs
(defn <get-geolocation []
(js/Promise.
(fn [resolve reject]
(if-let [geo (.-geolocation js/navigator)]
(.getCurrentPosition geo
(fn [position]
(let [coords (.-coords position)
pos {:lat (.-latitude coords)
:lng (.-longitude coords)}]
(resolve pos)))
(fn [error]
(reject (js/Error. (str "Geolocation error: " error))))
(clj->js {:enableHighAccuracy false
:timeout 3000
:maximumAge (* 24 60 60 1000)}))
(reject (js/Error. "Geolocation not supported")))))))
Usage:
(try
(let [{:as pos :keys [lat lng]} (new (e/task->cp (await-promise (<get-geolocation))))]
(js/console.log "lat:" lng " lng:" lng))
(catch js/Error ex ...)
#2024-07-0712:02Dustin Getzyes this is the right direction#2024-07-0712:04Dustin Getzperhaps more idiomatic to wrap geolocation into a signal rather than sampling it only once, but in your case (game instance discovery) this is fine#2024-07-0712:06Dustin Getzregarding our naming conventions -
• in , /auth? Maybe a token in localStorage?
Not for security-sensitive app – just to identify users for about 30 minutes in case they lock their phone and websocket dies & reconnects.
Should I generate an auth token on server, store in localStorage on client & pass it in on component mount?#2024-07-0809:51braai engineerPut differently, is it possible to store a username cookie from Electric without reloading the page?#2024-07-0809:57braai engineerOh cool, I see I can set a cookie from JS, and it shows up on subsequent load in :cookies (never used JS-side cookies before):
(defn set-cookie! [key value]
; do we need to do some URI encoding here?
(set! (.-cookie js/document) (str key "=" value ";path=/; max-age=86400")))#2024-07-0809:59Tommy JollyboatI'm also working on party games. I use localStorage myself, but cookies have the advantage (or disadvantage) that they can expire after a certain time.#2024-07-0810:01Tommy JollyboatI store their first websocket ID in localStorage, then use that as a user ID#2024-07-0813:50grounded_sageYou can make an ID and put it in localstorage and a map on the server. I used this for a chat to get messages of the chat in realtime across tabs and server side streaming of AI generated text across all tabs.#2024-07-0909:38braai engineerHow should I deal with simultaneous client/server changes where navigating on the client cancels a server action?
E.g. the server mutation is cancelled by the client navigating away, but result of d/transact! is not serializable:
(dom/button
(dom/on "click" (e/fn [_]
(e/server
(prn "this prints:" player-id my-team-id)
(let [pid (e/snapshot player-id)
tid (e/snapshot my-team-id)]
(prn "this does not print because Navigate!. below cancels it:" pid tid)
(e/discard ; I expect this to run before or at same time as the e/client block below.
(d/transact! conn [[:db/retract pid :player/team tid]]))
(e/client
(r/Navigate!. ['/ [::landing]]))))))
(dom/text "Leave Game"))
but wrapping it in a clj function works (leave-team! contains the transact), even though I am not waiting for the result on client:
(DangerButton. (e/fn [_]
(e/server
(db/leave-team! conn player-id (e/snapshot my-team-id))
(e/client
(r/Navigate!. ['/ [::landing]]))))
"Leave Game")
Is there a nice way to tell the client to navigate only after the server command has executed, even when the result is not serializable?#2024-07-0909:55xificurCthe simplest trick to sequence effects is to wrap them in case
(case (db/leave-team! ..)
(e/client (r/Navigate!. ..)))
case runs the "else" branch only when the test value resolves to a non-exceptional value#2024-07-0909:56braai engineerFeels like there should an Electric or Missionary primitive for this#2024-07-0909:56xificurCwrapping in a clj(s) function is preferred when possible#2024-07-0911:11braai engineerWhy does clicking the submit button on this form clear the ui/input, but pressing Enter on the form (which submits the form – same handler), commits the tx but does not clear the ui/input until I blur it?
(e/client
(let [!text (atom "")
text (e/watch !text)]
(dom/h1 (dom/text "Term Editor"))
(dom/form
(dom/on "submit" (e/fn [e]
(.preventDefault e)
(let [text (e/snapshot text)]
(e/server
(case (db/tx! conn
[{:db/id "new-term"
:term/text text}])
(e/client (reset! !text "")))))))
(ui/input text (e/fn [v] (reset! !text v)))
(dom/button ; clicking this clears the ui/input, but not pressing Enter to submit form.
(dom/props {:type "submit"})
(dom/text "Add Term")))))#2024-07-0911:32xificurCthat's how we designed ui/input. The first argument, in your case text, can be an arbitrary source that changes under your feet while you're typing in the input, which is bad UX (imagine you typing away and an update clobbers the text as you type)#2024-07-0919:55braai engineertext = (e/watch !text). By passing in V!, I expect the ui/input to call that when the value changes, and I call (reset! !text new-text). So then I expect ui/input to update when I intentionally clobber !text.#2024-07-0922:52xificurCui/input listens for focus and blur events and disregards the update while focused. If that doesn't fit your use case you should use dom/input directly#2024-07-0919:49braai engineerUX problem with optimistic writes: given a set of 5 checkboxes (each with a unique ID) viewed by many but controlled by one user, I would like the checked state to respond instantly on the controller side and propagate checked state to viewers. How do I achieve this without duplicating components?
I tried populating a client-side atom on the client with the initial set of checked IDs from the server and then swap! on change while simultaneously transacting to server db, which should flow back into the canonical set for all viewers (incl. controller), but it doesn't seem to improve the responsiveness, e.g.
(e/server
(let [ids [1 2 3 4 5] ; e.g. from DB
selected-ids (qry-checked-ids db)]
(e/client
(let [!checked-ids (atom (set selected-ids))
checked-ids (e/watch !checked-ids)]
(e/for-by identity [id ids]
(let [checked? (get checked-ids id)]
(dom/button
(dom/on "click"
(e/fn [_]
(e/client (swap! !checked-ids (if checked? disj conj) id))
(e/server (d/transact! conn [[(if checked? :db/retract :db/add) :checked-set id]]))))
(dom/text "click to check id" id))))))))
Why would controller's client not respond instantly?#2024-07-0921:41Tommy JollyboatBizarre. Is there still lag if you comment out d/transact!? Or perhaps there's a way to lumber the DB with an artifically long delay & confirm that that's the source of the lag#2024-07-0921:42Tommy JollyboatAlso - I may be wrong, but I think the much-anticipated release of Elec v3 and/or ui5 may offer new patterns for optimistic updates, so this may be a form of premature optimisation (assuming you can wait a little)#2024-07-0921:59Dustin Getzi suspect the Pending thrown by the transact could be doing surprising things, try silencing it. When v2 does surprising things wrt lag and undesired unmounting - it is usually a pending interaction, particularly in the body of e/for#2024-07-0923:38VincentYou can achieve this by tightening the pattern into an e/for-by#2024-07-1006:57braai engineer@U055PQH9R4M what do you mean tightening into an e/for-by? It is already an e/for-by.#2024-07-1008:18braai engineer@U058CTJ4P2L yes, if I add (Thread/sleep 3000) before the tx, it lags 3s longer.
If I e/offload the server-side tx!, the lag goes away and the client updates instantly, but I can see a re-render (button animation pops again) when the server updates.#2024-07-1008:34braai engineer@U09K620SG to squash the Pending, I should (try ... (catch Pending ex)), right?#2024-07-1009:55xificurCblocking calls block the thread running electric, suspending the entire runtime#2024-07-1009:56xificurCthat's why wrapping in e/offload works, it runs the blocking call on another thread so the electric runtime can continue#2024-07-1014:59VincentOh! I have been passing around db entity maps, not db-ids, I think that has enabled us to avoid the re-rendering of the button in this case#2024-07-1015:00VincentAn early design decision that had a lot of downstream benefits#2024-07-1010:00braai engineerHow to know when a component is fully rendered on this client? When a player presses a button to start a timer, some cards with terms should show. Problem is, the player presses the button and the cards only render a second or two later, putting them at a disadvantage.
How can I only start the timer when I know the cards are fully mounted and visible to the player? Some kind of on-mount event?
Or maybe use :visibility "hidden" | "visible" instead of conditional rendering?#2024-07-1010:02Dustin Getzwhat is the cause of the delay#2024-07-1010:05braai engineerI guess mounting of cards or sending of card data to client? The data is already available but I only render the card when time starts. Presumably sending it over when card is drawn (happens up-front), but using :visibility should solve?#2024-07-1010:09braai engineerI'm seeing about 150ms round-trip latency to Fly's Johannesburg region (very bad) on my mesh WiFi connection, but only 50ms on 4G. Should be like 20ms.
(doubt that adding more memory to machine would lower latency)#2024-07-1010:52Dustin GetzOk, yeah you'll need to work around Electric's deficiencies by preloading the data. Electric currently emits quite disoptimal network (e/for in particular creates undesired round trips). Electric v3 is the starting point for solving that. In the meantime you can try to flatten the amount of client/server nesting by lifting IO "to the top" (i.e., move entire e/for blocks to the client so that everything it might need is local before the e/for runs). Also, yes, manipulating visibility is helpful#2024-07-1010:59braai engineerFixed by using :visibility "hidden" on client 🙂#2024-07-1014:35braai engineerThinkFast, the word-guessing party game is now playable at https://thinkfast.fly.dev/
Written in 1152 lines of Electric Clojure. “Find Nearby Games” sorta broken rn, but you can scan the QR code to join or copy link to invite others to your game.
Currently everyone can see the card in play - will fix so players on your team can’t see it. Enjoy the mario easter egg if you wait out a full round with sound on :).
Will add more terms and category-based expansion card decks, e.g. “Formula 1”, “80s Music,” etc.
Always surprised at how much logic goes into modelling the state of even a 'simple' board game.#2024-07-1015:01Tommy JollyboatVery nice!
• Obviously the "Nearby Games" part is more of a placeholder currently, but it could do with filtering to show only games with active players, sorting by distance, and mapping to list the players in each game.
• Needs instructions in the "Geolocation not available" message, to say how to switch it on#2024-07-1015:02Tommy JollyboatWhat are you using for the radial timer?#2024-07-1015:06Tommy JollyboatGameplay-wise, I haven't tried with people, but my instinct would be to add more time, and decrease over subsequent rounds. alternatively, less terms, and increase the number#2024-07-1015:08braai engineer@U058CTJ4P2L yeah was thinking of calculating a handicap for strong players, e.g. "Tommy has a -3 second handicap," or to balance teams with stronger & weaker players.
In ZA the difficulty goes up every round because we play it as a drinking game.#2024-07-1015:37Tommy Jollyboat*DrinkFast#2024-07-1119:31braai engineerFixed "Find Nearby Games": https://thinkfast.fly.dev/(:thinkfast.main!nearby)/
although it takes you straight to the Play screen instead of lobby. Should take you to Lobby, but need more state because game could have already started. Just press "< game lobby" to join a team. Will fix.
It will find any games created in last 2 hours within 5km of you. Now show's game owner's player name.#2024-07-1409:10braai engineer@U058CTJ4P2L for donut chart timer, see .pie CSS at /thinkfast.css#2024-07-1517:46Tommy JollyboatThanks! I decided in the end to just use linear dom/progress , but I might come back to the donut later 🙂#2024-07-1606:40braai engineer👍:skin-tone-2: didn’t know about the <progress> element#2024-07-2322:22braai engineerThinkfast is now https://explaingame.com/ – playable at http://explaingame.com (much better domain than the Fly dot io one).
• Built over 2 weekends in ~1,600 lines of Electric Clojure. No other tech stack other exists today in which I could have built a real-time, multiplayer "board game" in this time frame, with this level of polish.
• The goal of ExplainGame is to get your teammates to say the words on your card before the timer runs out, using only word association clues, i.e. you as the explainer cannot say the words on the card.
• Fixed repeating words bug, where seen words were appearing again before players ran out of cards (words can only repeat after you run out of cards)
• Took 1st step toward monetization: users are now prompted to buy an expansion card deck when they run out of free cards. Will add cool categories like Cars, Golf, Rugby, etc. Still working on payment stuff.#2024-07-3015:36braai engineerposted "Show HN: I made an explanation party game in Electric Clojure" (ExplainGame)
/me dons scaling goggles#2024-07-3015:57braai engineerHN link: https://news.ycombinator.com/item?id=41110158#2024-07-3016:20Tommy JollyboatExcellent! Well described. In the UK, and possibly elsewhere, the popular published version of the game is called Articulate. It's fairly well-known, and might help explain the concept (or be worth including in metadata or whatever)#2024-07-3016:28braai engineerthx @U058CTJ4P2L, edited with mention of Articulate.#2024-07-3017:35Tommy JollyboatI'd leave more comments and upvotes, but I'm monstrously busy & don't have an account. Hopefully someone else will 😃#2024-08-0509:41Olav FosseI like the monetisation approach. Maybe try selling decks for other languages too?#2024-07-1018:55braai engineerHow do I turn off the exception logging when a client disconnects their websocket connection because they closed the tab? For me, this is expected, e.g. logs are filled with:
2024-07-10T18:53:57.854 app[e286765db13686] jnb [info] ERROR hyperfiddle.electric-ring-adapter: Websocket error
2024-07-10T18:53:57.854 app[e286765db13686] jnb [info] java.nio.channels.ClosedChannelException: null
... +20 more lines
#2024-07-1018:59Dustin Getzlogback.xml#2024-07-1019:26Vincentyeah control all that via logback#2024-07-2322:22braai engineerThinkfast is now https://explaingame.com/ – playable at http://explaingame.com (much better domain than the Fly dot io one).
• Built over 2 weekends in ~1,600 lines of Electric Clojure. No other tech stack other exists today in which I could have built a real-time, multiplayer "board game" in this time frame, with this level of polish.
• The goal of ExplainGame is to get your teammates to say the words on your card before the timer runs out, using only word association clues, i.e. you as the explainer cannot say the words on the card.
• Fixed repeating words bug, where seen words were appearing again before players ran out of cards (words can only repeat after you run out of cards)
• Took 1st step toward monetization: users are now prompted to buy an expansion card deck when they run out of free cards. Will add cool categories like Cars, Golf, Rugby, etc. Still working on payment stuff.#2024-07-3015:36braai engineerposted "Show HN: I made an explanation party game in Electric Clojure" (ExplainGame)
/me dons scaling goggles#2024-07-1115:25grounded_sageIt appears that my usage of drag and drop in clj-chatbot-ui was broken with the electric update.#2024-07-1115:26grounded_sage(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]))
;; Saving this file will automatically recompile and update in your browser
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
;; Draggable item
(dom/div (dom/props {:draggable true
:style {:width "150px"
:height "150px"
:background "red"}})
(dom/h1 (dom/text "Hello from Electric Clojure")))
;; Drop zone
(dom/div (dom/props {:style {:width "300px"
:height "300px"
:background "blue"}})
(dom/on "dragenter" (e/fn [e]
(.preventDefault e)
(println "drag enter")))
(dom/on "dragover" (e/fn [e]
(.preventDefault e)
(println "drag over")))
(dom/on "drop" (e/fn [e]
(.preventDefault e)
(println "drag drop"))))
(dom/p (dom/text "Source code for this page is in ")
(dom/code (dom/text "src/electric_start_app/main.cljc")))
(dom/p (dom/text "Make sure you check the ")
(dom/a (dom/props {:href "" :target "_blank"})
(dom/text "Electric Tutorial"))))))
This is the code I am testing on latest electric-starter and I can not seem to get "drop" event listener to be triggered.#2024-07-1115:29grounded_sageI may simply be doing something wrong but I cannot see it at present.#2024-07-1115:30grounded_sage@U051SPP9Z I know you have drag and drop going. Are you using the html drag and drop api?#2024-07-1115:46VincentYour code looks good to me at first read-thru. As a preliminary sanity-check 😅#2024-07-1115:57grounded_sageIt seems to me like a super subtle bug in the event handlers.#2024-07-1116:01grounded_sage#2024-07-1116:08Vincenthm. does altering the javascript load sequence help#2024-07-1116:21grounded_sageFor a moment I thought you might have been onto something. As I changed the ordering of the electric functions setting up the event listeners.#2024-07-1116:22grounded_sageHere I demonstrate that I am only able to trigger the "drop" event listener set up by electric by having a "dragover" event listener in plain javascript. Which is just strange behaviour.#2024-07-1116:23braai engineer@U05095F2K yeah, drag & drop is tricky. Are you dealing with mobile or desktop or both? For mobile I ended up rolling my own: listening for mouseup/`mousedown` & touchstart/`touchend` events with mousemove and then checking if the distance moved from starting point exceeds some threshold (e.g. 10px) and then (reset! !dragging? true).
I use z-index + (js/document.elementFromPoint x y) with :data-* attributes to check what components were tapped/clicked. So not really using the drag & drop API anymore.
But you can also impl. own collision detection if you know the location and size of all elements (but DOM can do that for you).#2024-07-1116:24grounded_sage@U051SPP9Z yes this would be my workaround. But there is a drag and drop html API there that is somewhat functional and I had been using it before to drag conversations into folders. So a little stumped at this behaviour I have at present.#2024-07-1116:26braai engineercould be an event listener state problem if the page did not fully reload. addEventListener persists if you don't removeEventListener on unmount#2024-07-1116:27grounded_sageI always do shift-cmd-R for my reloads#2024-07-1116:28braai engineerhmm, do you need (.preventDefault e) in your dragstart event?#2024-07-1116:29grounded_sagethat actually drops the drag it seems#2024-07-1116:31grounded_sageI know the browsers have given us these API's so we don't have to implement from scratch but seems like we generally end up doing this anyways because of some edge cases and weirdness#2024-07-1116:32grounded_sageAlso to answer your question @U051SPP9Z desktop at present but would want it to work with both.#2024-07-1116:34braai engineer@U05095F2K Claude suggests: try to set some dataTransfer on dragEnter and dragOver to hint that the drop zone remains a valid target throughout the drag operation, e.g. (I haven't tested it yet):
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
;; Draggable item
(dom/div (dom/props {:draggable true
:style {:width "150px"
:height "150px"
:background "red"}})
(dom/h1 (dom/text "Hello from Electric Clojure")))
;; Drop zone
(dom/div (dom/props {:style {:width "300px"
:height "300px"
:background "blue"}})
(dom/on "dragenter" (e/fn [e]
(.preventDefault e)
(set! (.. e -dataTransfer -dropEffect) "move")
(println "drag enter")))
(dom/on "dragover" (e/fn [e]
(.preventDefault e)
(set! (.. e -dataTransfer -dropEffect) "move")
(println "drag over")))
(dom/on "drop" (e/fn [e]
(.preventDefault e)
(println "drag drop"))))
(dom/p (dom/text "Source code for this page is in ")
(dom/code (dom/text "src/electric_start_app/main.cljc")))
(dom/p (dom/text "Make sure you check the ")
(dom/a (dom/props {:href "" :target "_blank"})
(dom/text "Electric Tutorial"))))))#2024-07-1116:34braai engineerdunno why this would help tbh#2024-07-1116:36grounded_sageYea I rubber duck with ChatGPT a lot. Also played around with this as an idea.
1. I had it working with a prior version of electric
2. Consulting MDN docs it doesn't appear to be a requirement
3. It works in the code I have in these videos... but only when I set up another event listener in plain javascript. Which seems to trigger the other event handler in electric somehow.#2024-07-1116:37grounded_sageThis is non-critical and non-blocking for me so I can work around it. Just very strange to me.#2024-07-1116:37braai engineerdoes it work if you manually call (.addEventListener dom/node "...") from Electric?#2024-07-1116:39braai engineerYou can enumerate event listeners on an element with getEventListeners to log them#2024-07-1116:41grounded_sageCan also find them here#2024-07-1116:42grounded_sage.addEventListener also seems to fail for me. This is a put it aside moment for me. If I really need drag and drop I may just implement a custom solution.#2024-07-1116:43grounded_sageCan do more with animations and UI then anyways#2024-07-1118:00xificurCIf addEventListener doesn't work it seems like a problem outside electric? Try to make it work with js in the console first#2024-07-1203:01grounded_sageIt works but only with the external JavaScript also attaching a “dragover” listener. #2024-07-1203:02grounded_sageI demonstrated this in the videos#2024-07-1207:58xificurCsorry I did not watch them. Try swapping dom/on for dom/on!, which takes a cljs fn for the callback. Electric's dom/on might skip handling some events since it runs in continuous time, in which case preventDefault doesn't run on the event#2024-07-1209:46braai engineerdoes dom/on! call removeEventListener on unmount?#2024-07-1210:12xificurCSure#2024-07-1207:51braai engineerAny way to reduce the Docker image size? my little Thinkfast game produces an 817MB Docker image, which only has stock Electric deps + Datalevin.
Maybe ditch JDK base image and only use JRE for final image?#2024-07-1208:52VincentHmm. Uberjar will be closer to the "real" or "necessary" size, but not sure what to do to make docker less dock#2024-07-1209:21Dustin Getzwhy do you care, is this a metered internet costs issue?#2024-07-1209:43braai engineercould maybe run on smaller memory machine ($)? Not just for this project but others. And faster deploys possibly.#2024-07-1214:13JAtkinsthis might be too much added work, but I've had great success with Nix for clojure projects. Since you define your own base image, you can slim down docker images quite a bit. For some flavor of what nix+electric looks like, I've got a nearly complete deployable img here: https://github.com/JJ-Atkinson/personal-rss-reserver/blob/main/flake.nix#2024-07-1216:58Nathan SmithAnybody tried using https://github.com/wolfi-dev/? My org started requiring these for all images we publish.#2024-07-1216:07braai engineeradded another component and am running out of heap space when trying to deploy to Fly:
#21 25.73 shadow-cljs - nREPL server started on port 45653
#21 25.73 [:prod] Compiling ...
#21 69.20 {:optimize true, :debug false, :verbose false, :hyperfiddle.electric/user-version "95f48ad-dirty"}
#21 98.85 Exception in thread "clojure.core.async.timers/timeout-daemon" java.lang.OutOfMemoryError: Java heap space
#21 98.85 at java.base/java.lang.Long.valueOf(Long.java:1180)
#21 98.85 at clojure.lang.Numbers.num(Numbers.java:1844)
#21 98.85 at clojure.core.async.impl.timers$timeout_worker.invokeStatic(timers.clj:48)
#21 98.85 at clojure.core.async.impl.timers$timeout_worker.invoke(timers.clj:43)
#21 98.85 at clojure.lang.AFn.run(AFn.java:22)
#21 98.85 at java.base/java.lang.Thread.run(Thread.java:829)
#21 99.13 RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Exception parsing "thinkfast/main.cljc"
#21 99.13 com.google.javascript.jscomp.PrebuildDependencyInfo.prebuild (PrebuildDependencyInfo.java:82)
#21 99.13 com.google.javascript.jscomp.Compiler.maybeDoThreadedParsing (Compiler.java:2302)#2024-07-1216:51braai engineersplit up code into multiple namespaces to try fix#2024-07-1216:53braai engineersplitting up namespaces seems to have fixed it#2024-07-1708:24braai engineer@U09K620SG is this compiler memory limit an issue with shadow or Electric? Can it be fixed? Bit scary that I need to refactor all my namespaces after fixing a bug for a hotfix that then hits a memory limit. (which is what happened here)#2024-07-1219:13Nikolas PafitisHas anybody used R2DBC with electric/missionary? It looks like it could be a great fit for end to end reactivity.#2024-07-1223:17Vincentnot sure what it is 😄#2024-07-1223:18VincentXTDB zero setup time but more wordy database/write.s
datomic needs a special apparatus to give you the latest db over a missionary flow ...
R2DBC has realtime updat?#2024-07-1311:21Nikolas Pafitiswhich XTDB version are you talking about? what do you mean more wordy?#2024-07-1311:23Nikolas PafitisR2DBC is a specification/api, like JDBC that provides a fully reactive non-blocking api. There's a handful of drivers for most of the mainstream sql databases. https://r2dbc.io/drivers/
Don't know exactly how it works though never used it. I'm just trying to evaluate whether or not fits my needs#2024-07-1223:18VincentI have an idea for an app we sort-of need:
Hi-Res Bootlegs
What is a bootleg? It's a personal recording of an event's audio, like a show or concert or performance...#2024-07-1223:19VincentOpen source app probably!
Because can't figure out how to money this app idea 😅
But:
one person puts their phone on the speaker at the venue and gets the best sound recording.
other people record the band playing for varying intervals
we all upload it to one timeline thing, and create a "Hi-Res bootleg" with great audio and changing scenes/clips when/if we have video.#2024-07-1223:21Vincentwould be a nice use-case for the multiplayer-abilities of electric. it's non-trivial to summon FFMPEG and encode things as a finished product, and it's non-trivial to process uploads and generate thumbnails and make a timeline mechanism from elemental electrick bits, but could be:
+ amazing way to master electric
+ has real-world use-cases more numerous every day
+ would look good on your CV#2024-07-1401:01Tommy JollyboatYeah, it very much violates copyright law#2024-07-1401:06Tommy JollyboatObviously a lot of people film gigs and put them on YouTube, but that's more a case of flying under the radar. It's also not always popular with the acts, but that varies by Industry (and country, no doubt). You're definitely not encouraged to film at comedy/theatre shows in the UK; bands are generally more relaxed.#2024-07-1401:08Tommy JollyboatAnyway, this is getting off-topic#2024-07-1418:01VincentHuh? Artist allow tapings#2024-07-1418:02VincentClearly this would be for where artists and venues allow you to record#2024-07-1418:02VincentI thought that went without saying#2024-07-1418:03VincentIN THE CASES WHERE YOU'RE ALLOWED TO RECORD, such a service would be very helpful for making Hi-Res bootlegs. Some bands allow bootleg recordings. A lot of them do. Of course, it's not the sound board output, but it's cool. Weaving them together could be cool. Also this could simply be a group video editing program at its base. Anyway, engineering was my question here, not if the squad car will come take me away 😆#2024-07-1416:27Drew Verleewhats the ideal way to display the results of rfc tests in a clerk notebook? with-out-str?#2024-07-1416:52Dustin Getzyou should be able to redirect stdout right? Nextjournal works great with rcf OOTB : https://nextjournal.com/dustingetz/missionary-relieve-backpressure#2024-07-1416:53Dustin Getzfailing that, there are reporter extension points - see source code#2024-07-1416:58Drew Verleein my clerk notebook, the code from that example, doesn't return the test results to the rendered page, it returns nil (see picture)
ill try to look at the source code for inspiration.#2024-07-1417:11Dustin Getzit's displaying the expression result nil without the prints#2024-07-1417:12Dustin GetzAre they going to your terminal? The prints must be going somewhere#2024-07-1417:12Drew Verleethere going to the cider-repl buffer.#2024-07-1601:28Dustin GetzHey all it's time to start pumping up our twitter account - please send electric hype, memes etc if you are succeeding with electric! (thank you @petrus @v1nc3ntpull1ng for beating the drum you guys are great)
you can tag either https://x.com/ElectricClojure or https://x.com/dustingetz , also i intend to start posting content on linkedin https://www.linkedin.com/in/dustingetz/ please follow or connect - leave a note if you connect#2024-07-1607:07braai engineerElectric needs a cute logo to stand out on the socials. Something electrifying. Few concepts.#2024-07-1607:32braai engineereven faster#2024-07-1609:17Dustin Getzlol#2024-07-1615:21Tommy JollyboatHow about an "e" with an electron orbit, something like:#2024-07-1616:00Tommy JollyboatThis also isn't really a serious suggestion, but I had a fun idea of S(erver) containing C(lient)#2024-07-1618:51VincentYou guys are such jokers lmao 😂
Obviously the logo should be what Rich Hickey's hair looks like, electrified#2024-07-1619:37Dustin Getzloooool#2024-07-1700:03Tommy JollyboatI promise I'll stop eventually... But I realised there's a good visual synergy between the react logo + the yin-yang of lisp / clojure#2024-07-1700:06Tommy JollyboatWhich admittedly does still also look a bit like IE, and might give older web developers PTSD#2024-07-1708:21braai engineerwhen it comes to good logos, don’t try to be clever. figure out what message you want to convey and search for a natural abstraction that carries the brand in a memorable way. Something that would make a good favicon. Avoid tropes that would be confused with Electron or nipples.#2024-07-1608:10henrikTo pile on some fun Electric examples, here’s a prototype AI research agent running in Electric.#2024-07-1612:23stagmooseMay i ask how to make this working:
I would love to call .getCurrentTime for an youtube video, and get time as fast as possible and sync that time with a backend atom
is this possible?
thanks for your help!
main.cljc
(ns app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?(:cljs ["youtube" :as YT])))
(e/defn Player []
(e/client
(let [player-div (dom/div (dom/props {:id "player"} (dom/text "test")))
player (YT.Player. "player"
(clj->js {:height "360"
:width "640"
:videoId "M7lc1UVf-VE"
:events {"onReady" (fn [event]
(js/console.log "Player is ready"))
"onStateChange" (fn [event]
(js/console.log "Player state changed"))}}))]
player-div
(e/on-unmount #(println "unmount"))
player)))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(e/client
(let [player (Player.)]
(dom/div "video cur time: " (dom/text (.getCurrentTime player))))
(dom/div (dom/text "client time: " (e/client e/system-time-ms)))))))
shadow.cljs
{:builds
{:dev {:target :browser
:devtools {:loader-mode :default, :watch-dir "resources/public/app"}
:output-dir "resources/public/app/js"
:asset-path "/js"
:modules {:main {:entries [dev] :init-fn dev/start!}}
:build-hooks [(hyperfiddle.electric.shadow-cljs.hooks/reload-clj)]
:js-options {:resolve {"youtube" {:target :global
:global "YT"}}}}
:prod {:target :browser
:output-dir "resources/public/app/js"
:asset-path "/js"
:modules {:main {:entries [prod] :init-fn prod/start!}}
:module-hash-names true}}}
index.html
#2024-07-1708:28braai engineermost compelling marketing case for Electric is rapid development of production-ready enterprise software that drives costs down 10x with 1-2 devs, which is what Dustin has been punting on socials.
Live-coding screencasts of building serious software would do well. “Let me just add this feature quick and deploy it.” 15 mins later
“And there we go.”#2024-07-1716:52Dustin Getzour main problem is investors don’t care about anything with the word Clojure in or around it#2024-07-1716:53Dustin Getzwe’re trying to get enough UI stuff paved out (on v3) to start talking about hyperfiddle again which is declarative UI#2024-07-1808:45braai engineerDrop the “Clojure.” Just “Electric.” It’s cleaner#2024-07-1708:38braai engineerWould be cool if my Cursive IDE “coloured” the Electric DAG visually so that e/server branches were a different shade from e/client branches.
This would also help teach Electric. #2024-07-1708:44xificurCPOC for emacs here https://gitlab.com/xificurC/hf-electric.el/-/blob/master/hf-electric.el?ref_type=heads#2024-07-1709:05Absolute NegativityI've always wished there's scope highlight in general like http://community.schemewiki.org/?sicp-ex-3.38, for Emacs.#2024-07-1800:34grounded_sage@U0ETXRFEW #2024-07-1810:20pez@U05095F2K if it can be solved in some general manner it would fit in Calva. Otherwise it’s better suited for a #C03DPCLCV9N script. (Or we could say that Joyride is the general solution 😀 )#2024-07-1817:26grounded_sage@U0ETXRFEW yea I don’t really feel a need for it. Was just tagging you in for awareness of tooling conversation.#2024-07-1710:14nakkayaI have pattern in my app that is,
• An image is uploaded via a POST request.
• When POST returns image is added to a list of images in UI.
This part is easy to handle from electric. What I am trying to structure is on the backend, image goes through a bunch of workflows via temporal. This can take between 1 second to 2 minutes depending on the image. When temporal is done I would like to update some tables on the UI. Currently I am handling this using an atom on the backend and a watch on the front end, when backend completes atom is updated and frontend updates automatically. Is there a better way to handle this sort of situation? I feel like this method has a tendency to turn into mess as the number of workflows increase. @dustingetz @xifi#2024-07-1716:51Dustin Getzhow big of a problem is it today? a first class temporal integration will require some design work. it seems related to the LLM streaming completions problem, which electric v3 has better primitives for#2024-07-1812:11nakkayaToday not much. I was just wondering if there are better idioms to handle this using missionary or something else.#2024-07-1814:17Dustin Getzsurely there are, we will work them out next year#2024-07-1712:22Deniz YilmazokHi I have question regarding the application environment.
That is, I have variable called datomic-conn this variable both declared in dev.cljc and prod.cljc .
Those two files provide connections to different datomic databases.
But in my code I'm using (binding [conn @(requiring-resolve 'dev/datomic-conn)] which uses variable declared in the dev.cljc .
As a result, application crashes when it is ran in prod.cljc config.
How can I solve this problem ? For example, can I detect the config of the application at the runtime. So shat, I can call requiring-resolve conditionally ?
Like: (if (= env "dev")
requiring-resolve 'dev/datomic-conn
requiring-resolve 'prod/datomic-conn)
Or do you suggest different method.
Thanks.#2024-07-1713:14nakkayaIs there a particular reason to pick which db to use in prod.cljc and dev.cljc . You can have you db calls in a separate namespace and have the logic in there to pick which db to connect that way you get a single db handle to load for both prod and dev . Don't know about datamic but this is how I handle postgres connections prod, dev and test.#2024-07-1713:18nakkayathis will also fix resolving things in runtime.#2024-07-1713:22Deniz YilmazokWe have two files because each file connects to a different db urls (setups)
dev.cljc connects to a "datomic:<dev://localhost:4334>"
prod.cljc connects to a "datomic:<sql://db-name?jdbc:postgresql://>"#2024-07-1713:29nakkayaMove the logic to select which JDBC URL to connect to a different namespace, in there have your (= env "dev") logic pick the correct URL and open the connection. From the electric side just use db/conn it should have already configured for the right env when the ns loaded.
(ns db)
(def conn-url (if (= env "dev") dev-conn prod-conn))
(def conn (some-connection-pool conn-url))
Something along the lines of above. Then conn will have correct connection to db based on env . This is how I do it.#2024-07-1713:34Deniz YilmazokI'm also looking for a method to detect context at the runtime
Do you know how can I detect the config ?
My conditional check (= env "dev")
was an imaginary example.#2024-07-1713:35Deniz YilmazokIf I can solve the detection
your (ns db) solution becomes plausible#2024-07-1713:38nakkayaSet an env variable for prod, when env variable is not set default to dev . Then you imageinary check becomes (= (System/getenv "APP_ENV") "prod")#2024-07-1713:44Deniz YilmazokThanks @U5H4U2HEH I will try.#2024-07-1713:22Deniz YilmazokWe have two files because each file connects to a different db urls (setups)
dev.cljc connects to a "datomic:<dev://localhost:4334>"
prod.cljc connects to a "datomic:<sql://db-name?jdbc:postgresql://>"#2024-07-1714:08stagmooseMay i ask why this simple app is not working?
the page in my browser is not even showing the "0"
(ns app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?(:cljs [goog.object :as gobj])))
#?(:clj (def !x (atom 0)))
(e/def x (e/server (e/watch (!x))))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(e/client
(dom/div
(dom/text "atom from server: " (e/server (str x))))
(ui/button (e/fn []
(e/server (swap! !x inc)))
(dom/text "inc"))))))
the error:
ERROR hyperfiddle.electric: #error {
:cause class clojure.lang.Atom cannot be cast to class clojure.lang.IFn (clojure.lang.Atom and clojure.lang.IFn are in unnamed module of loader 'app')#2024-07-1714:24Absolute NegativityI think you want (e/def x (e/server (e/watch !x))).#2024-07-1714:27Dustin Getz(e/watch (!x)) is a typo, you want (e/watch !x)#2024-07-1714:27stagmooseah, sorry for this silly mistake:rolling_on_the_floor_laughing:#2024-07-1714:27stagmoosetook me awhile#2024-07-1811:39braai engineerSome progress on my wood counter using Electric with OpenCV. Lots of tuning, preprocessing and contour filtering still to do.
(Claude 3.5 has been very helpful in deciphering Java class usage)#2024-07-1812:33braai engineerHow do I wait for the result of (e/offload #(do-long-running-io! ...) m/blk), since I can't do e/client or e/server signals in the function to tell the client to refresh something after the IO is done?#2024-07-1813:32Hendrike/offload throws pending until the first result is available. so there should be no need to await something.
there is also an e/offload-task that throws pending again, if something in the form changes:
https://clojurians.slack.com/archives/C7Q9GSHFV/p1710067056603109?thread_ts=1710013148.958439&cid=C7Q9GSHFV
Is this what you need? If not, what exactly do you want to achieve?#2024-07-1813:33braai engineerI want to reset! an atom on the client when the offloaded task has completed on the server.#2024-07-1814:00Hendrik(e/defn Foobar []
(e/client
(let [!processing? (atom true)
processing? (e/watch !processing?)]
(println processing?)
(e/server
(let [r (e/offload #(do (Thread/sleep 1000)
42) m/blk)]
(e/client
(when r
(reset! !processing? false)
(e/on-unmount #(reset! !processing? true)))))))))
I would guess something like this should do it?#2024-07-1814:20braai engineerAh cool - didn’t realize I could wait on the offload result r. r would have to be non-nil tho (in ur example)#2024-07-1814:37HendrikIf you expect nil values, then you can work around it by defining a sentinel value#2024-07-1814:53braai engineerWhat is a sentinel value, (e/offload #(do (nil-thing!) true))?#2024-07-1816:40grounded_sageOh I like this. @U023LKF2PQV thank you!
I have been having the client have a let binding with (e/server watch.. on a predefined server atom as the value. But this is much more direct. #2024-07-1816:56Hendrike.g. wrap it in a vec :
(let [r (e/offload #(let [nilable-result (nil-thing!)][nilable-result]))] (when (vec? r)....)
#2024-07-1819:37xificurC(case r (foo)) only needs a non-exceptional value for (foo) to run#2024-07-1815:40JAtkinsDoes electric have a stack limit? #2024-07-1819:20Vincentversion one yes
version two yes, but rarely encounter it after Incremental Compilation
version three not sure#2024-07-1820:56telekidIs @U055PQH9R4M’s answer true for both compile time and run time, or just compile time?#2024-07-1821:47Dustin Getzelectric (runtime) strictly speaking doesn’t have a call stack, i think you’re asking about any limitations to call depth, i don’t recall if we currently have a theoretical limit - at one point we hit a low level javascript lambda depth ceiling but that was fixed long ago#2024-07-1821:48Dustin Getzif you have an error please post it#2024-07-1906:49henrikI think there practically is one, but I’ve only encountered it when I’ve used Electric functions HOF-style.
I wrote a ring-like abstraction at one point in order to handle “booting” the app. This means a stack of “middleware”, where each middleware receives next one, and calls it once it has done its work. This broke down with obscure errors.
The exact same stack, but “static” (i.e., the call to the next handler is inlined in the function body of the previous one) worked fine.#2024-07-1823:00grounded_sageWhat is the most straightfoward way to get values from (chan->ap chan) . I currently have a channel on the backend and I want those values rendered on the frontend.#2024-07-1900:40Dustin Getztry m/reductions to add an initial value and then use new to get an electric value out of it#2024-07-2217:33VincentDiscovered that in Electric if you have the serverside code already writ for the given arity and therefore your serverside refs do not need a compile then... you can rewrite front-end code all day and have it recompile all day no issues.
(defn win [] ...)
(defn slam [] ...)
(e/defn Componante []
(let [...]
(e/client
(ui/button (e/fn [] (e/server (slam)))))))
;; we can change to (win) and the front-end will recompile in dev mode, and work once the javascript is [re]loaded without need for a server restart
#2024-07-2217:33Vincent(obviously this is quite a claim and I would like others to verify it)#2024-07-2217:34Dustin Getzi am confused, are you developing with your dev repl connected to prod server?#2024-07-2217:35Vincentthis is dev mode#2024-07-2217:35Dustin Getzin a normal dev setup, hot code reloading should be fairly seamless and not require server restarts#2024-07-2217:35Vincentawesome. i guess it's something i was not really used to. it makes perfect sense but i just kinda put the dots of the constellation together#2024-07-2217:37Dustin Getzok#2024-07-2217:38Dustin Getzto be clear the server will reload also every time you save a cljc file with electric code in it (there is a shadow-cljs hook that does this)#2024-07-2220:37VincentIs there a way I can watch a standard file?#2024-07-2220:38Vincentbasically have some sort of edn file i want to slurp ... ought that automatically absorb file changes? I haven't tried it yet, another program (python) will save to the .edn file#2024-07-2221:31Dustin Getzyou need missionary and a file watcher library to subscribe to file system events from the OS#2024-07-2221:32Dustin Getzleo implemented this a few yrs ago i’d have to dig around, try searching missionary channel #2024-07-2222:07nakkayaI gave it a shot but could not find, this would also help me for my use case as well if either one of you find it can you share it. Thanks.#2024-07-2303:33VincentThere's one called hawk 😄
https://github.com/wkf/hawk#2024-07-2303:37VincentI dunno how to do this watch file stuff but I am on the right track I think.#2024-07-2309:16nakkayayeap, without missionary. I would have the watcher watch the file and update an atom with the content of edn then you can watch the atom from front end. That is what I do currently to wait for async events. I am wondering if there are better idioms using missionary.#2024-07-2309:31Dustin Getzyes, the main thing to improve from that point is to diff the file contents in some way (by line or structure) and send diffs through the DAG for efficient view updates. we will need v3 for that#2024-07-2716:01Dustin GetzHere is a missionary POC. This is absolutely unsupported, you are on your own#2024-07-2716:04Dustin GetzI also recall issues with MacOS having terrible performance through the standard Java interfaces, which Hawk and others might fix#2024-07-2716:04Dustin GetzI don't even think it returned fully correct results, i recall it like missing events#2024-07-2716:04Dustin GetzYou could also look at the shadow-cljs source code and see what they use#2024-07-2716:09VincentVery cool. Yeah, in my use-case I need to get data from other providers and store it. And i'm thinking a flat file might work nicely. But, if it's spotty, maybe gotta go db route#2024-07-2317:24nakkayaIs it possible to call a server side function in e/on-unmount . When component is shown it start connections to couple of end points to stream data into an atom it needs to run while the component is in view but when component unmounts I would like to make a call to close and cleanup connections.#2024-07-2318:15xificurCWouldn't (e/server (e/on-unmount ..)) work?#2024-07-2319:12nakkaya:man-facepalming: that works I tried the other way around, (`e/on-unmount (e/server ..))`#2024-07-2318:56AHMET OĞUZHAN ENGİNHello! I am working with tables in Electric and need to learn how to sort by table cell and implement the pagination feature. Does anyone have a code snippet or a basic example that I can study? Thanks!#2024-07-2323:40Dustin Getzthe datomic browser example is the strategy we use in our projects - see the source code in contrib.gridsheet ns i believe in the electric core github repo. You will likely end up forking it#2024-07-2323:42Dustin Getzthe strategy is to reuse the same 20-30 row elements, keeping them static and absolutely positioned, and writing to their cell contents as you scroll#2024-07-2402:52stagmooseif i have this main.cljc file:
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?(:clj [datalevin.core :as d])
[electric-starter-app.db :as db]))
how can i make the modification inside db.cljc take effect without rerun clj -A:dev -X dev/-main
my db.cljc look like this:
(ns electric-starter-app.db
(:require #?(:clj [datalevin.core :as d])))
#?(:clj (do
(def schema
{:user/name {:db/valueType :db.type/string}
:user/email {:db/valueType :db.type/string
:db/unique :db.unique/identity}
:user/videos {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many
:db/ref :video}
:video/url {:db/valueType :db.type/string}
:video/user {:db/valueType :db.type/ref
:db/ref :user}
:video/ab-loops {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many
:db/ref :ab-loop}
:ab-loop/start {:db/valueType :db.type/long}
:ab-loop/end {:db/valueType :db.type/long}
:ab-loop/video {:db/valueType :db.type/ref
:db/ref :video}})
(defonce conn (d/get-conn "data/db" schema))))#2024-07-2501:13stagmoose@U09K620SG this is my short recording demo my problem
you can see that after i save the new db.cljc, it has shown the new "build completed"
am i doing wrong setup here
thanks for your help!#2024-07-2501:15stagmoosebut the result in the repl is the wrong old value#2024-07-2501:21stagmoosemy shadow-cljs.edn:
{:builds
{:dev {:target :browser
:devtools {:loader-mode :default, :watch-dir "resources/public/electric_starter_app"}
:output-dir "resources/public/electric_starter_app/js"
:asset-path "/js"
:modules {:main {:entries [dev] :init-fn dev/start!}}
:build-hooks [(hyperfiddle.electric.shadow-cljs.hooks/reload-clj)]}
:prod {:target :browser
:output-dir "resources/public/electric_starter_app/js"
:asset-path "/js"
:modules {:main {:entries [prod] :init-fn prod/start!}}
:module-hash-names true}}
:nrepl {:port 42000
:middleware []}}
deps.edn:
{:deps {com.hyperfiddle/electric {:mvn/version "v2-alpha-540-ga4699532"}
ring/ring {:mvn/version "1.11.0"} ; comes with Jetty
org.clojure/clojure {:mvn/version "1.12.0-alpha5"}
org.clojure/clojurescript {:mvn/version "1.11.121"}
org.clojure/tools.logging {:mvn/version "1.2.4"}
ch.qos.logback/logback-classic {:mvn/version "1.4.14"}
datalevin/datalevin {:mvn/version "0.9.8"}
applied-science/js-interop {:mvn/version "0.4.2"}
io.github.tonsky/clj-reload {:mvn/version "0.7.0"}}
:paths ["src" "resources"]
:aliases {:dev
{:jvm-opts ["--add-opens=java.base/java.nio=ALL-UNNAMED"
"--add-opens=java.base/sun.nio.ch=ALL-UNNAMED"]
:extra-paths ["src-dev"]
:extra-deps {thheller/shadow-cljs {:mvn/version "2.26.2"}
io.github.clojure/tools.build {:mvn/version "0.9.6"
:exclusions [com.google.guava/guava ; Guava version conflict between tools.build and clojurescript.
org.slf4j/slf4j-nop]}}} ; clashes with app logger
:prod
{:extra-paths ["src-prod"]}
:build ; use `clj -X:build build-client`, NOT -T! build/app classpath contamination cannot be prevented
{:extra-paths ["src-build"]
:ns-default build
:extra-deps {thheller/shadow-cljs {:mvn/version "2.28.11"}
io.github.clojure/tools.build {:mvn/version "0.9.6"
:exclusions [com.google.guava/guava ; Guava version conflict between tools.build and clojurescript.
org.slf4j/slf4j-nop]}}} ; clashes with app logger
}
}
#2024-07-2501:30stagmooseforgot to show the main.cljc in the video:
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]
#?(:clj [datalevin.core :as d])
[electric-starter-app.db :as db]
#?(:cljs [goog.object :as gobj])
#?(:cljs [goog.functions :as gf])))
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(ui/button (e/fn []
(e/server
(db/testing-fn)))
(dom/text "testing-fn from db.cljc")))))#2024-07-2516:51VincentIt's looking at it like a normal clojure serverside namespace#2024-07-2516:52VincentCan we watch multiple .cljc files for serverside changes? I don't know if this is supported.#2024-07-2516:53VincentMore specifically, let's find the Electric code that is doing the hook. Thinking the hook specifically checks out main.cljc -- more dot cljc namespaces might need more hooks#2024-07-2516:54VincentIs one hook 🪝 per .cljc reasonable?#2024-07-2501:06cjmurphyIs it true there's no reason you can't use a traditional back-end with Electric (https://clojurians.slack.com/archives/C03S1KBA2/p1721869175774969?thread_ts=1721840151.892639&cid=C03S1KBA2)?#2024-07-2509:23Dustin Getzyes, and also v3 (upcoming version) has experimental first class support for single peer systems as well as systems where a peer disappears and comes back#2024-07-2510:07cjmurphyI'm wondering if I'll be able to get the same functionality as with an Android or ~MacOS~ iOS App, on a cellphone/Tablet device. So if there's no internet but everything required is there otherwise (the JavaScript cached and say some data been saved in LocalStorage) then the application (URL request) should be able to be started up without saying "Unable to connect to the internet". I do realise that a URL request when the internet is down should give such a message! Maybe a local proxy or something... Just trying to think of a way of not re-writing the FE for 3 targets (iOS, Android and Electric on the browser).#2024-07-2516:50Vincent@U0D5RN0S1 because everything in SPA land is oriented around navigation based on the URL bar href.location ,, it might be worth intercepting this on "network requests" in the context of mobile [go into a tunnel]#2024-07-2517:58cjmurphyThanks for thinking about it @U055PQH9R4M, and yes some kind of interception will be needed.#2024-07-2520:42VincentYeah, maybe we can think about having 2 datastores: datascript on the front-end as a backup, and datomic on the backend. or maybe like a battery SCSI -> datomic talks to datascript talks to app. if datomic is unavailable you still have datascript#2024-07-2704:14cjmurphyFor me a client-side (FE) database can just be a data structure, no need for Datascript. It can even be normalised as Fulcro does it, if needed. Updates would go to both databases (FE and BE). The watch could be just on the FE database. A use case here would be someone taking orders from customers on a Tablet. Perhaps they have to walk far from the WiFi. But obviously no saving on the FE here, which was what I was alluding to as an Android or iOS App having certain advantages.#2024-07-2517:06VincentFor those brave warriors who have Electric'd just clientside (no server) can someone explain the deployment process in a succinct way?#2024-07-2517:13Dustin Getzv2 requires a server and live websocket, to run it without that requires a custom entrypoint which is experts only#2024-07-2517:13Dustin Getzv3 has better capabilities here, i'd wait#2024-07-2607:03grounded_sageexcited for electric being more capable in this direction#2024-07-2522:23VincentHey, iOS native wrapper over electric app, what do we think?#2024-07-2606:57grounded_sageI've thought about this a bit with Swift. I did one for desktop but have not gone any further.#2024-07-2618:21JAtkinsIs there any important reason to use git describe --tags --long --always --dirty instead of the sha of the current git commit as electric user version during prod deploys?#2024-07-2618:24Dustin Getznot that i recall#2024-07-2618:25JAtkins👍#2024-07-2619:44Dustin GetzElectric v3 first look! https://hyperfiddle-docs.notion.site/Electric-v3-teaser-improved-transfer-semantics-735b10c3a0dc424e93e060a0a3e80226 - please ask questions#2024-07-2620:32refsetreads very well and makes a lot of a sense! 👏
small typo:
> is ~a~ brittle
are there other reasons people might still reach for macros with Electric? or has undesired transfer really been the only significant culprit?#2024-07-2620:53Dustin Getzelectric-dom still uses macros for inverted control flow - e.g. in (dom/div (println 1)) the div mounts before the println, which is the opposite of regular Clojure control flow. So users wanting dom-style container syntax for their own widgets will need macros for that unless someone figures out a good hiccup-style syntax that makes the problem go away. Beyond that IDK, we're still figuring it out#2024-07-2620:54Dustin GetzA lot of electric-dom internals are no longer macros, particularly event handling stuff can be electric fns now#2024-07-2622:22Piotr RoterskiGreat, well motivated choices. :first_place_medal:
Splitting e/for-by into e/cursor + e/diff-by feels very promising for giving more explicit control collection transfer over network. I would love to hear more details on those two!
Question 1:
e/diff-by gets passed identity as the first arg, so I assume it's something like pred / compare function - what arguments does it receive? What could we do with it? Is it for paginating longer lazy collection into chunks that are fed to the cursor? Or is it more about caching? I see filtering is done by the second arg function.#2024-07-2622:26Piotr RoterskiQuestion 2:
e/Token is mysterious - is it related or similar to dom/node? Is it analogous to React ref?#2024-07-2622:33VincentAbout halfway through. Reads nicely. Very cool to see undesired transfer addressed and the way it is addressed is beautifully stated. The pattern ought not alter the mechanisms underneath, when the pattern is too wordy or extra wordy. So yeah, I like what you've said so far.
Regarding the use of $ ... uh
Okay
Um. I guess new made sense somehow. Uh, $ hrm. I dunno. Maybe it's just another thing to get used to. For whatever reason, the reification aspect of new and Component. made more intuitive sense to me. Seeing ($ Inc ..) is a little like, arr there be jQuery ahead.
But, maybe something like (Electric InputBox []) might not be all that crazy or whacky at this point
Regarding e/cursor and e/diff-by what if I don't want to use brain on these things? Can electric brain for me?#2024-07-2622:37Piotr RoterskiQuestion 3:
dom/OnAll aggregates events over the dom/node so they can be consumed by e/cursor. Seems like a perfect place for stuff like throttling and debouncing. Am I reading this right?#2024-07-2622:38VincentBased on the example, it looks like e/cursor is the iterator. Which is cool, i just did not understand that from the name.#2024-07-2622:39VincentI think it's smart that diff and gathering steps are separated, I wonder if there might be more obvious names however I could also get used to e/cursor#2024-07-2622:44Piotr Roterski@U055PQH9R4M I don't mind $ function, seems familiar coming from uix. I read cursor more like pagination cursor so it feels fitting#2024-07-2622:45Dustin Getzi also loved the ‘new’ syntax vs $, we will see where we end up. it is very hard to get good error messages when you misuse new. We can also get weird and :refer [E] … (E Foo) 😈 #2024-07-2623:03Vincent(E. ContactsList) seems kinda natural to my eyes at first glance
So you have the diffable-set or the progressively-filled-set and you have the render-by-item thang or like render-an-set-of-itams kinda a vibe...
(e/by-item [id (e/diffable-set ...)]) or is it too craze#2024-07-2623:09VincentThanks for the blog post, the sneak peek, and the logic behind the design decisions. Super cool to see it in action. I bet it took a while! I'm really glad it has come together so elegantly.#2024-07-2623:09VincentIn theory I should be able to write very nested code that all compiles the same because Electric won't be fooled by my phantom nesting. We'll put that to the test 😼#2024-07-2623:16VincentOne last comment: the more I can inline the better. So for example, I really enjoy e/for-by because it's just one, and I can fill all the extra fluff as args.
I kinda have this idea that at its core at electric value should just render what it has in it when summoned. And then maybe like a differential-electric-value might actually be invisible-able from me. A theory.#2024-07-2700:05grantIn the third paragraph, “Many users have been willing to accept…“.#2024-07-2700:09grantIn the fourth paragraph, “…transfers a value over the network…”#2024-07-2701:41Tommy JollyboatVery nice! One initial thought - while it's all well & clearly written, the problem statement of "what was wrong in v2" is interesting to those of us that use v2, but could probably be shortened if the intended audience is people new to Electric, since it's describing a soon-to-be-obsolete problem.#2024-07-2701:43Tommy JollyboatI like the (E.) syntax, adjusted fast, and rarely make mistakes but having to switch to new for dynamic e/fn values and threading is a little awkward. A single syntax and better error msgs are strong arguments for ($ E)#2024-07-2702:00Tommy JollyboatI'd also love an explanation of diff-by and cursor. Cursor seems a lot like e/for, and could perhaps be called e/each, but maybe I should understand it before renaming it 😉#2024-07-2702:02Dustin Getze/each has been proposed, the argument for e/cursor will become more clear when we explain diff-by which is 🤯 #2024-07-2702:03Dustin Getzi hated the name e/cursor at first but it’s growing on me - an important part of this is how much we want to conceal or lean into the differential nature of electric v3#2024-07-2708:49henrikFor $ there is prior art in UIx, where it’s used like ($ :div …) or ($ MyComp). As a former UIx user, it makes sense to me as well, but I realize that’s biased.
We’ve written our own $ macro on v2 to get the UIx style. I.e, it goes further by supporting all dom elements as keywords, implicit dom/props when written ($ :div {:class …}), and most controversially I’m guessing, support for ($ :on …) and ($ :on! …).#2024-07-2712:13Dustin Getzre e/Token I am not satisfied with the API yet, it has some overlap with dom/OnAll which also involves a callback so let me focus there instead. Here's a quick explainer of the underlying structure of OnAll; the callback is used to terminate the extended event#2024-07-2712:15Dustin GetzSo imagine rapidly mashing a button that transacts on the server, all those txns are concurrent and progress independently, which is why we need to use e/cursor (iterator) with OnAll to express the multiplicity#2024-07-2719:10noonianI suspect the answer is (understanably) no, but is it possible to combine v2 and v3 at all and migrate an electric application gradually? Or will one need to break everything until it's fully re-written to use the v3 syntax?#2024-07-2813:05Dustin Getzdeep compatibility is not a goal and we do not intend to test. However upgrading Electric code has been mechanical so far - mostly deleting over-complicated client/server machinery, and turning macros into functions, which feels really good. Imperative code (e.g. the internals of a typeahead or virtual scroll that is listening to various browser events) has also been straightforward because the clojure/script functions are unchanged, missionary code listening to browser events is unchanged, and the electric/missionary interface has only changed in a superficial way. It feels really promising.#2024-08-0311:54bdbrodieLate to the thread, but I’ll throw in my $0.02
This release is perhaps a the best time to introduce one last set of name changes, optimized for the huge number of new users who are going to discover Electric in 2025. (E. ) looks great. e/cursor reminds me of the ActiveRecord pattern of years ago.
I think the current draft announcement does a good job of lowering the upgrade fear for current v2'ers like me. But I agree with Vincent’s point. Maybe another doc written specifically for the large number of new people and don’t want to have the cognitive load of “history”, makes sense.#2024-08-0312:13Dustin GetzGot it - yeah, I'll do a Electric for brand new people post soon#2024-07-2623:23VincentThere's a library called Hawk that leverages the java file watcher thang.
Hawk can be used with duratom to make watchable atoms that save to the file system...
Databases are great, but maybe for rapid iteration it's worth a look.#2024-07-2706:26Olav FosseI thought atoms are watchable by default#2024-07-2706:33Vincentatoms are#2024-07-2706:38Olav Fossehmm, so what is Hawk for? afaik u can just watch a duratom ootb#2024-07-2721:07VincentElectric is fun to work with. I have been building a lot of new applications with it. Well, not new exactly, but ideas I've had for a few years and never had the right framework to make it fun to make and also extendable in some way. Functionality-wise. Feels good. Still coming to terms with the newfound power. Let's not shy away from the intensity of the light!⚡ 😎#2024-07-2722:05Tommy JollyboatWell said! Until February 2023, I was trying & failing to build my project with a lot of JS, 3 Svelte clients, and something that abstracted over websockets. Then found first Hyperfiddle and then Electric, and almost instantly realised it's the answer.
My progress has been slower than yours, but I was new to web apps and Clojure (but not Lisp). I watched a lot of talks, struggled a lot with choosing & learning libraries, databases, architectural patterns, clojure tooling, hosting, WLAN networking, etc... and I've rewritten from the ground up twice. I've gone back on a lot of decisions, but never even considered switching out Electric.
I'd also never used a database before. Can recommend. They're a classic for a reason.#2024-07-2812:15braai engineerhow many Fly machines should I temporarily scale my Electric app to and which edge regions to use for a US-centric ProductHunt / Show HN listing?
main annoyance is replicating Datalevin data to each edge node because volume is machine-bound.#2024-07-2812:23Dustin Getzthat is not a question we can answer. The only electric app that ever hit the front page is the electric y combinator essay, which stayed up, but is not interactive and the only thing it did on the server was read the file system #2024-07-2812:24Dustin Getzi think it had 50 concurrent users for 1-2 min sessions#2024-07-2812:56Dustin Getzoh and we used big fly instances in 5 regions, it cost a few hundred bucks for a couple days#2024-07-2812:58braai engineerthanks – that helps. I haven't done any benchmarking. I don't expect my app to do more numbers than your essay and I'm not doing anything DB intensive in ExplainGame.
I see that Fly has a concurrency soft_limit and hard_limit. Assume I need to raise these?#2024-07-2812:58braai engineerDustin, do you have if any "rule of thumb" ideas about how many concurrent users Electric can handle on Fly, memory-wise or CPU wise before you run into problems?#2024-07-2813:00braai engineerPut differently: have you measured how much additional memory pressure (in KB?) each concurrent user places on an Electric backend instance? Or is that affected by component complexity?#2024-07-2813:10Dustin Getzwe have not measured anything and it likely depends on your app, you should measure#2024-07-2813:13Dustin Getzthe cost of an empty websocket is low, there are blog posts about scaling websockets on other platforms such as liveview. One really big box can get you into 10,000s of connections. Above that you will have the same ops challenges that other websocket based platforms have - which you will have money to solve from your millions of monthly active users resulting in 10,000s of concurrent connections#2024-07-2813:14Dustin Getzbeyond the incremental cost of a websocket connection, the rest is up to what your app is doing per connection#2024-07-2813:15Dustin Getzone liveview shop hit 100k+ connections per box, you can find details on hacker news in one of the electric discussions#2024-07-2813:31VincentAwesome. 100k+ is a new figure, I was estimating at least 55k+ 😃#2024-07-2813:31VincentSounds like maybe can do with one fly instance? i guess geographically centralized for your userbase?#2024-07-2813:31Vincent(if database replication is a task)#2024-07-2813:31Vincent(why do it)#2024-07-2813:33braai engineerlatency to US East vs West Coast and UK#2024-07-2814:00Dustin Getzi also don’t think fly is the right architecture for single box database apps, we use fly because our demos are viewed globally#2024-07-2814:01braai engineerin my case Fly is a good fit because http://explaingame.com is a social party game played in the same room so each game state can live on the nearest edge, as long as the card database is replicated to all nodes#2024-07-2814:19Dustin Getz> as long as the card database is replicated to all nodes#2024-07-2814:21braai engineerby cards or deck I mean the terms used to populate freshly drawn cards in a given hand, which is a long list that is practically read-only - could even be hardcoded.#2024-07-2905:30Vincentread-only, yeah that makes sense#2024-07-2817:59weiis there a better way to write this?
#?(:cljs (defonce !last-tick-client-time (atom 0)))
#?(:cljs (defonce !last-tick-server-time (atom 0)))
(e/def last-tick-server-time (e/watch (e/client !last-tick-server-time)))
(e/def last-tick-client-time (e/watch (e/client !last-tick-client-time)))
(e/def ms-since-last-tick
(e/client
(when (not= last-tick-server-time server-timestamp)
(reset! !last-tick-server-time server-timestamp)
(reset! !last-tick-client-time e/system-time-ms))
(- e/system-time-ms last-tick-client-time)))
i'm writing a game that runs on server ticks (~1 second per tick), and it's useful to have the time since the last tick for client-side interpolation#2024-07-2818:33Dustin Getzin missionary (m/reductions -), in electric (- a (delay a)) idk if we include a delay operator try searching missionary channel for discussion of delay#2024-07-2818:33Dustin Getzor this channel#2024-07-2909:30braai engineer@U066TMAKS I do something similar in BraaiSim where energy transferred to food items = heat * time-elapsed, so I have a (defn next-state [db time-elapsed] (let [last-tick (:last-tick query-db)] ...)) fn that returns DataScript txes. Caller passes current rounded time to (handle-frame! conn time) fn that runs every 100ms, i.e. pass in time as (* 100 (round (/ e/system-time-ms 100.0))) and transacts new state from my entrypoint component, while the game is in a running state (not paused).#2024-07-2917:16weithanks for the answers! think i have it sorted 🙏#2024-07-2915:03stagmooseis there a way to make sure the execution order in electric?
if i have a function like this:
(e/fn [looping?]
(when looping?
(let [!cur-loop-idx (atom 0)
cur-loop-idx (e/watch !cur-loop-idx)]
(play-a-section. player (get ab-loops cur-loop-idx))
(when (checker (< 500 (mod c 1000)) #(outside? player (get ab-loops cur-loop-idx)))
(swap! !cur-loop-idx (fn [x]
(mod (inc x) (count ab-loops))))))))
can i make sure (when (checker ...)) always happens after an electric function play-a-section is called first?
i want to start playing an interval (seek to the start of the interval) then check whether current player time is inside the interval
in the current behavior, if the first interval is outside, it will inc the idx, then start playing, which is not what i want
is it possible to add some sleep between them?#2024-07-2915:07stagmoosetried something like:
(e/fn []
(let [t (m/sp (t/debug "1")
(m/? (m/sleep 1000))
(t/debug "2"))]
(-> (js/Promise. t)
(.then #(t/debug % "then"))
(.catch #(t/debug % "catch")))))
the above will wait and execute in order if act as on-click function of a buttom
but if i put var like player inside the m/sp , it will complain the player var is undefined, but the player is just before the m/sp in the let binding#2024-07-2918:04Tommy JollyboatI've not really understood the question, but you can delay with something like
(let [2min (* 1000 60 2)
target (e/snapshot (+ e/system-time-ms 2min)))]
(when (< t (- e/system-time-ms))
:do-stuff-after-two-minutes))#2024-07-2918:05Tommy JollyboatAlso it's good to stick to the convention of capitalised names for e/defn's, (Play-a-section. ,,,)#2024-07-2918:23Tommy JollyboatAnd it might not help with delays, but you can hack seqeuncing effects with case, it's a bit ugly: https://clojurians.slack.com/archives/C7Q9GSHFV/p1720518934034849?thread_ts=1720517922.808089&cid=C7Q9GSHFV#2024-07-2918:24Tommy Jollyboatif all this is way off the mark, I'm sure someone will provide a more helpful answer soon 🙂#2024-07-2918:29xificurCIn (case x y) y runs after x resolves to a non-exceptional value, which often fits the bill#2024-07-2918:30xificurCThe simplest and cheapest way to sequence effects is to call them in a clj(s) fn#2024-07-2923:43stagmoose@U058CTJ4P2L thanks for your reply!
i tried your idea:
(let [five 5000
target (e/snapshot (+ e/system-time-ms five))]
(when (< target e/system-time-ms)
(t/debug :five)))
this works if put inside a electric component
however, if put as onclick function of button, any idea why it will not work?
i would expect it will print after i click the button
(dom/button
(dom/on "click"
(e/fn [e]
(let [five 5000
target (e/snapshot (+ e/system-time-ms five))]
(when (< target (+ e/system-time-ms))
(t/debug :!!!!! :five)))))
(dom/text "testing")
(dom/props {:class "bg-red-300"}))
#2024-07-3011:13xificurCThe dom/on callback unmounts before that. If you really want to wait there's a contrib.electric-contrib ns with a macro called after#2024-07-3011:14stagmooseah it makes sense now. thanks for your help~#2024-07-3017:55Vincent"Javascript video editing, in the browser" https://news.ycombinator.com/item?id=41108843#2024-07-3017:55VincentOne year of development for the SDK^#2024-07-3017:55VincentWho is crazy now#2024-07-3023:02VincentHaha very funny. Yes I'm still crazy.
But also, making a multiplayer distributed video editing app is not so insane a notion! People have been hard at work. And they did not have Electric 😼#2024-07-3108:25braai engineerSomething for those hoping to scale Electric on http://Fly.io with different DB volumes on each edge node w/o DB replication: I scaled up 3 edge nodes (JNB, LAX & LHR), each with its own Datalevin DB yesterday, under the assumption that users from the same location would hit the same edge node, but some http://ExplainGame.com players in France have eSIMs in their phones, so they were routed to a different edge and naturally, the host's game data was not present on that edge. Obvious in retrospect, but I was hoping Fly would figure out the locations.
Thankfully, distributed data replication is simple and easy.
This has me considering using the "host" as the source of truth and streaming data to clients via a global bus :thinking_face:. Don't really want to go full Datomic for this, and would like to avoid a central DB since game data is highly shardable.
Sigh. Last time I tried Fly's distributed Sqlite thingy (LiteFS) it was a nightmare to get working.
Is there a Datalog implementation that can use SQLite as a backing store? I imagine :db/id might be a problem.#2024-07-3109:38markaddlemanNot a direect answer to your question but Datascript has pluggable durable storage backends now https://github.com/tonsky/datascript/blob/master/docs/storage.md
I don’t think there’s a SQLLite backend, though#2024-07-3109:51braai engineerThanks. The SQL Storage backend supports :dbtype :sqlite: https://github.com/tonsky/datascript-storage-sql#2024-07-3109:58markaddlemanOh cool#2024-07-3116:11grounded_sageDatahike can do this with it’s JDBC library#2024-08-0111:58braai engineerUnfortunately LiteFS does not proxy Websocket requests, so LiteFS does not work with Electric Clojure yet: https://clojurians.slack.com/archives/C7Q9GSHFV/p1722512968579539?thread_ts=1722509123.373529&cid=C7Q9GSHFV#2024-07-3113:14Dustin Getzhttps://hyperfiddle-docs.notion.site/Electric-progress-update-July-2024-9429836036e4489fb9ce3dc9ddf7faee. Old news for most of you. Scroll to the bottom for some info about Electric v3.#2024-07-3113:42henrik> electric lexical/dynamic scope is now unified, no more duplicated bindings (i.e. existing on both client and server separately)
🤯#2024-07-3113:46Dustin Getzyes you are going to like this upgrade 🙂#2024-07-3113:52henrikI knew that, but I’m starting to get a sense of how much#2024-08-0204:24Dave MaysExciting stuff.
A single peer / clojurescript only mode is mentioned available for v3. Could you help me understand the value in this? I was thinking Electric's main value proposition was helping out at the client server boundary (drastic oversimplification), and so if you're in clojurescript / client only what is there for Electric to do? Or is this a step towards an offline-first version?#2024-08-0208:04Dustin Getzit’s just a react replacement at that point#2024-08-0208:05Dustin Getzstronger composition model than react, and the effect based rendering model is very good#2024-08-0208:44jazzytomatovery exciting stuff. What's the upgrade path gonna look like from v2 to v3? would it introduce a lot of breaking changes?#2024-08-0214:31Dustin Getzhttps://clojurians.slack.com/archives/C7Q9GSHFV/p1722171955046349?thread_ts=1722023085.164749&cid=C7Q9GSHFV#2024-07-3113:45henrik#2024-07-3113:49Dustin GetzAh yes, Leo needed a bit more extra time to work out the design#2024-07-3123:19Olav FosseJust got started with Electric today and having great fun. I played around a bit and hacked up some code to filter some posts based on an input field: (anymentions/mention-stream-posts …). Unfortunately my filtering is very slow so I’d like to debounce my code. I think debouncing the tms edge pointing to (mention-stream-posts (d/db …) HERE …) might be best.
How can I do this?#2024-07-3123:22Olav FosseOops never mind the e/for-by, this is my working code:#2024-07-3123:53Dustin Getzi don’t remember if we provide electric denounce check electric core source code file it might be there. otherwise search here and on #CL85MBPEF for discussions#2024-07-3123:54Dustin Getzalso offload the slow query to a thread pool with e/offload to not block the async computation on the server, i doubt it solves your problem but it will prevent other problems later#2024-08-0110:46braai engineer@U027R2XBQEP if you are already using a global dynamically-bound Electric e/def db for your db value, you can pass that db value to your query, so you are not asking for a fresh DB on every query.#2024-08-0110:57Olav FosseHmm, I’m not using e/def db#2024-08-0212:12Olav FosseAlso, how can I go about rendering posts one by one as they’re filtered? I’m flexible with regards to the interface/implementation of anymention/mention-stream-posts.#2024-08-0212:13braai engineeremit (PostComponent. post-id) in e/for-by for your query result#2024-08-0214:32Dustin Getze/for-by - https://electric.hyperfiddle.net/(electric-tutorial.demo-system-properties!%53ystem%50roperties)/#2024-08-0509:33Olav FosseHello again and happy Monday! I looked at the demos at http://electric.hyperfiddle.net, but I don’t quite think it applies in my case because I want to render from a discrete flow rather than a seq.
Concretely I have a function ms:filtered-posts which returns a discrete flow fpf of posts. The filtering is slow so ms:filtered-posts will transfer posts one by one. I’d like to render fpf by appending to the div one by one as the posts are transferred.
I have two ideas for how to solve this:
From my understanding e/for(-by) takes a seq, so I could transform fpf into a lazy seq. I don’t like this idea, because I’m going flow->seq->flow just to shoehorn fpf into e/for-(by).
My other idea is to m/relieve fpf to get a continuous flow cfpf containing(what’s the right nomenclature here?) a seq of the filtered posts so far. cfpf I can transform to an Electric value and render as is.
How should I go about solving this? Thanks!#2024-08-0509:39xificurCif fpf is a discrete flow returning posts one by one, in v2 you can do
(e/for-by identity [post (new (m/reductions conj [] fpf))]
(dom/li (dom/text post)))
#2024-08-0512:09Olav FosseI can’t get that to work. Ftr fpf is a flow on the server.#2024-08-0512:13xificurC(e/server
(e/for-by identity [post (new (m/reductions conj [] fpf))]
(e/client (dom/li (dom/text post)))))#2024-08-0512:14Olav FosseI’ll try that 👍#2024-08-0512:19Olav FosseThis renders, but it’s batching
(e/server
(let [fpf (anymentions/ms:filtered-posts
(assoc tms
:filtering-mechanism :sfilter
:threshold 0.5)
(d/db anymentions/conn))]
(e/for-by identity [post (new (m/reductions conj [] fpf))]
(e/client (dom/li (dom/text (:reddit-post/title post)))))))#2024-08-0512:23Olav FosseFor context see this message, I should’ve mentioned that in today’s message :D#2024-08-0512:32Dustin Getzyour query may be blocking the electric server thread#2024-08-0512:34Dustin Getzwhat exactly do you mean by "it's batching"#2024-08-0512:36Dustin Getzyou can confirm electric's behavior by getting reductions over an atom that has a collection in it, and updating the atom from the repl to conj more items one at a time#2024-08-0512:39Dustin Getzfwiw, for-by performs diffing (a O(n) scan that compares the current collection value and previous collection value to determine the changeset) upon each change to the collection. It is not fast enough for large collections, you'll want to keep it to 20-30 rows for display (i.e. one page of results). Electric v3 has a way to ingest change streams like this natively (without realizing the collection only to immediately diff it again - which is silly and pointless)#2024-08-0513:12Olav FosseBy “it’s batching” I mean that the posts don’t appear one-by-one as they’re transferred from the flow, but rather they’re all rendered at once. I understand now that’s how for-by works.#2024-08-0513:16Dustin Getztry adding m/relieve after m/reductions to ensure that the consumer process is decoupled from the producer process#2024-08-0513:17Dustin Getzthe video also doesn't prove that the producer isn't batching#2024-08-0513:17Olav FosseOh, I already added that#2024-08-0513:18Dustin Getztry slowing down the producer#2024-08-0513:18xificurCm/relieve takes one more argument#2024-08-0513:18xificurC(m/relieve {} (m/reductions ...))#2024-08-0513:19Olav FosseI believe {} is implicit#2024-08-0513:19xificurCTIL#2024-08-0513:20Dustin Getzhow many items are in that list?#2024-08-0513:21Dustin Getzthe for-by is expensive, if it can't keep up with the changes, it will keep starting over#2024-08-0513:21Olav FosseYes, I think for-by is the wrong way to go#2024-08-0513:22xificurCcan you share the missionary filter flow code for fpf as well?#2024-08-0513:23Dustin Getzif the for-by is too slow, iiuc in v2 the next best thing you can do is render out of band of electric (i.e. no electric-dom, you have to write low level missionary effects to manage the dom yourself). I think you're just playing around right? if so just wait for v3#2024-08-0513:30Olav FosseYes, here’s my newbie Missionary code, it’s blocks a lot 😅
(defn ms:filtered-posts [{:as ms :keys [sources]} db]
(->> (m/seed
(->> (d/qseq '[:find ?p ?title ?selftext ?permalink ?created
:keys db/id reddit-post/title reddit-post/selftext reddit-post/permalink reddit-post/created
:where [?p :reddit-post/subreddit ?sr!]
[(anymentions/normalize-subreddit ?sr!) ?sr]
[(contains? ?srs ?sr)]
[?p :reddit-post/title ?title]
[?p :reddit-post/selftext ?selftext]
[?p :reddit-post/permalink ?permalink]
[?p :reddit-post/created ?created]
:in $ ?srs]
db
(into #{} (map normalize-subreddit) sources))
(sort-by :reddit-post/created (comp - compare))))
(m/eduction (comp (map (inline-obj :reddit-post/selftext)) ;; This hits the disk, I could cache this!
(filter (partial forward-post? ms)) ;; forward-post? can also be slow as it does expensive math and/or hits openai
(map #(do (println "(hash emitted-post) = " (hash %)) %))))))
#2024-08-0513:31Olav FosseI’ll make the datomic query async#2024-08-0513:33Olav FosseYeah, this is just an internal tool that I thought would be a good fit for Electric. Most of my code is just HTMX.#2024-08-0513:39xificurCyou should convert all IO to missionary, this version will block at random points and block the entire thread electric is running on, stopping electric for that duration#2024-08-0513:47Dustin Getze/offload seems sufficient to stop blocking the electric server, but even still doesn’t qseq block until all the joins are computed? Datomic joins are not streamy#2024-08-0513:48Dustin Getzmeaning, it’s batching because it’s a batch query actually#2024-08-0513:49Olav Fosse.#2024-08-0513:50Olav FosseThe query is batched and then the filtering is streaming#2024-08-0514:04Olav FosseI'm used to thinking the filtering takes most of the time, but I recently made it 100x faster by switching from vectors to unboxed arrays for some math, so now it may actually be negligible.
In any case as showed in the video clip, posts are rapidly being emitted from the filter and then once the emitting is done (i.e no more messages are being printed in the terminal) they all show up at the same time in the browser. That's what I mean by batching.#2024-08-0514:05Olav FosseAnyways thanks a lot for helping me out! I have a lot to go off and study on my own now and then I'll check out the new v3 features when that time comes 😄 #2024-08-0514:22Dustin Getzthe posts are rapidly emitted in one immediate burst (likely separated by <1ms, and the terminal IO is the bottleneck, the elements can arrive faster than the terminal can print) and all this happens AFTER the joins complete. So the for-by sees a long pause, and then likely all the elements arrive all in one 16ms interval (while the terminal has queued the println IO at various points so the terminal IO is likely not even correlated with the sampling rate of the rendering process or the sampling rate of the websocket process)#2024-08-0212:12Olav FosseAlso, how can I go about rendering posts one by one as they’re filtered? I’m flexible with regards to the interface/implementation of anymention/mention-stream-posts.#2024-08-0509:33Olav FosseHello again and happy Monday! I looked at the demos at http://electric.hyperfiddle.net, but I don’t quite think it applies in my case because I want to render from a discrete flow rather than a seq.
Concretely I have a function ms:filtered-posts which returns a discrete flow fpf of posts. The filtering is slow so ms:filtered-posts will transfer posts one by one. I’d like to render fpf by appending to the div one by one as the posts are transferred.
I have two ideas for how to solve this:
From my understanding e/for(-by) takes a seq, so I could transform fpf into a lazy seq. I don’t like this idea, because I’m going flow->seq->flow just to shoehorn fpf into e/for-(by).
My other idea is to m/relieve fpf to get a continuous flow cfpf containing(what’s the right nomenclature here?) a seq of the filtered posts so far. cfpf I can transform to an Electric value and render as is.
How should I go about solving this? Thanks!#2024-08-0110:45braai engineerHas anyone gotten the electric-starter-app working with Fly's LiteFS? (in order to use DataScript with SQLite as backing store). It's not obvious from the https://fly.io/docs/litefs/speedrun/ if Java / Clojure is even supported.#2024-08-0111:49braai engineerThe LiteFS proxy does not support proxying secure websocket requests (`wss://`) at this time, so breaks for Electric.
https://fly.io/docs/litefs/proxy/#websockets
LiteFS issue:
https://github.com/superfly/litefs/issues/427#2024-08-0112:29weii have not, but that would be an great combination if it works#2024-08-0112:34braai engineerExplainGame https://www.producthunt.com/posts/explaingame, with a shout-out to Electric Clojure.#2024-08-0311:26braai engineerExplainGame is on Reddit /r/southafrica today (30 Seconds is a popular South African board game): https://www.reddit.com/r/southafrica/comments/1eizd3b/explaingame_the_party_explanation_game_30_seconds/
14 upvotes already! Single Fly JNB node is holding up fine so far :)#2024-08-0417:50Olav FosseI tried ExplainGame at a family gathering and it was great fun!
Some feedback:
• It would be nice to have an option for longer rounds. 30 seconds is a bit short for non-native English speakers.
• My sister fell out of the round, she managed to rejoin, but the old user stuck around. A user GC or mechanism to kick users would be sweet.
• The hosting/joining UX was great! The location based joining didn't work, but joining via QR code was easy.#2024-08-0417:51braai engineerthx for feedback! you can remove a player in the lobby but I think there’s a restart game bug#2024-08-0204:39Dave MaysWhat's the favorite way to deploy a production Electric based application? Say you were ready to deploy the Electric XTDB demo as a production site, is there a straight forward way to deploy to something like http://Fly.io? Do you deploy the database together with the application or separately?#2024-08-0208:03Dustin Getzdigital ocean is the most straightforward way - database and java app on same machine#2024-08-0208:03Dustin Getzgraduate to aws#2024-08-0209:23henrikWe’ve done this on Fly. DB and app on the same server, with their postgres thing as durable storage.#2024-08-0216:14grounded_sageI know this isn’t the priority of Electric but it would be super nice to be able to front end only build. I prefer Electric over React for developing the front end and sometimes “webapps” live inside of other ecosystems like Telegram mini apps. Contemplating a small gig but my only pause is having to step away from this experience ⚡#2024-08-0216:22markaddlemanI believe this is in the plan for Electric v3. Check out https://clojurians.slack.com/archives/C7Q9GSHFV/p1722431682180419#2024-08-0216:27grounded_sageIs this true @U09K620SG? I thought I saw you say it would be easier to do. But will it be possible without being hacky?#2024-08-0216:44Dustin Getzit is done#2024-08-0216:45Dustin Getzin principle #2024-08-0216:57grounded_sageHmm. I would like to test drive.#2024-08-0216:26VincentAnyone working on an internal knowledge thingy in electric clojure?
Some starting points from a thread in offtopic
https://github.com/nextjournal/clerk
https://github.com/zeitstein/brimm#2024-08-0217:33refsethttps://github.com/lumberdev/tesserae seems like a pretty neat starting point for such a thing (https://www.youtube.com/watch?v=nEt06LLQaBY&feature=youtu.be)#2024-08-0219:54mattiasI sketched out my thoughts on a PKM in this repo:
https://github.com/subspace-ai/subspace/tree/master
But tesserae looks pretty great, am thinking of basing v2 of subspace on tesserae. Will maybe sketch out the UI once electric v3 is out.#2024-08-0220:37VincentFunk yes that's right subspace looks like it's along the same lines. So cool!#2024-08-0220:37VincentSketch out the UI today in your notebook 😜#2024-08-0222:21refsetSome starting points from a thread in offtopicthanks for mentioning this thread! it's got a lot of good opinions and ideas in it. For ease of reference it's here:
https://clojurians.slack.com/archives/C03RZGPG3/p1722439345938229
And now I'm wondering whether Slack will add backlinks someday 😅
for more non-Electric-specific PKM chat feel free to use #tools-for-thought#2024-08-0219:01Tommyis there a way to get a missionary flow of new xtdb dbs while not using the in-process node? I want a persistent xtdb (lmdb backend) and to use electric with it. Thttps://github.com/hyperfiddle/electric-fiddle/blob/28b875d3eaa77f2d8b4716f383ba8f5ed2e1b92e/src/xtdb_demo/xtdb_contrib.clj#L6 still says it only works for in-process node#2024-08-0219:27Tommyjust tried it with lmdb node, and it worked fine! is this comment worth updating? did something change?#2024-08-0219:40refsetin-process != in-memory - LMDB counts as in-process too. The example for being out-of-process would mean using XT's HTTP stack (the server and client modules)#2024-08-0220:01xificurCNot writing that as (not= in-process in-memory) is just blasphemy#2024-08-0220:05refsetgosh, you're right...I've been writing so much SQL recently that I initially wrote in-process <> in-memory#2024-08-0220:26Dustin Getzwhat does "LMDB" mean#2024-08-0220:42refsetas in using the popular embedded key-value store LMDB as the index storage for XTDB v1 (and potentially also for tx-log + doc-store components), see https://v1-docs.xtdb.com/storage/lmdb/#2024-08-0300:25stagmooseI am reading the Typeahead example in v3 doc
(e/defn Typeahead [v-id Options OptionLabel]
(e/client
(dom/div (dom/props {:class "hyperfiddle-typeahead"})
(let [container dom/node
!v-id (atom v-id) v-id (e/watch !v-id)]
(dom/input
(dom/props {:placeholder "Filter..."})
(if-some [close! ($ e/Token ($ dom/On "focus"))]
(let [search ($ dom/On "input" #(-> % .-target .-value))]
(binding [dom/node container] ; portal
(dom/ul
(e/cursor [id ($ Options search)]
(dom/li (dom/text ($ OptionLabel id))
($ dom/On "click" (fn [e]
(doto e (.stopPropagation) (.preventDefault))
(reset! !v-id id) (close!))))))))
(dom/props {:value ($ OptionLabel v-id)}))) ; controlled only when not focused
v-id))))
for this code, i was wondering the purpose of let [container dom/node and binding [dom/node container], so i asked claude sonnet:#2024-08-0300:25stagmooseis this ai reply correct?:
The use of let [container dom/node] followed by binding [dom/node container] in this code serves a specific purpose:
1. let [container dom/node]: This captures the current DOM node (the div with class "hyperfiddle-typeahead") and binds it to the container variable. This is done outside the input's focus event handler.
2. binding [dom/node container]: This temporarily rebinds dom/node to the captured container within the scope of the binding form. This is done inside the input's focus event handler.
The purpose of this pattern is to create a "portal" effect. It allows the ul element (which contains the typeahead options) to be rendered as a child of the outer div, rather than as a child of the input element. This is often desirable for typeahead or dropdown components to ensure proper positioning and styling.
By capturing the outer div's node and then rebinding it within the focus handler, the code ensures that the list of options is rendered in the correct place in the DOM hierarchy, regardless of where the current focus might be.
This technique provides flexibility in DOM structure while maintaining the logical relationship between the input and its associated options list.#2024-08-0300:27stagmoosehttps://gist.github.com/dustingetz/0ecd64f41ded68477a1f2952eeb9b203#2024-08-0301:15Dustin Getzyes we want the options list to be managed by that if statement. there are other ways to write it#2024-08-0302:09stagmoosecan the binding dom/node be used inside a for-by block?
this works:
(e/defn Tab [videos]
(e/client
(let [!active-tab (atom (-> videos first :db/id))
active-tab (e/watch !active-tab)]
(dom/div (dom/props {:class "col-span-3"})
(dom/ul
(e/for-by :db/id [video videos]
(let [tab-id (:db/id video)]
(dom/li (dom/props {:class "bg-blue-100 hover:bg-red-100 mx-1 px-2 inline-block cursor-pointer"})
(dom/a (dom/props {:onclick (fn [_]
(reset! !active-tab tab-id))})
(dom/text tab-id))))))
(e/for-by :db/id [video videos]
(let [tab-id (:db/id video)]
(dom/div
(dom/props {:class (if (= active-tab tab-id)
"block"
"hidden")})
(PlayerV4. video))))))))
to eliminate the two for-by, rewrote to this:
but only the div for the last video is generated
(e/defn Tab [videos]
(e/client
(let [!active-tab (atom (-> videos first :db/id))
active-tab (e/watch !active-tab)]
(dom/div (dom/props {:class "col-span-3"})
(dom/div (dom/text "active-tab: " active-tab))
(let [content-container dom/node]
(dom/ul
(e/for-by :db/id [video videos]
(let [tab-id (:db/id video)]
(dom/li (dom/props {:class "bg-blue-100 hover:bg-red-100 mx-1 px-2 inline-block cursor-pointer"})
(dom/a (dom/props {:onclick (fn [_]
(reset! !active-tab tab-id))})
(dom/text tab-id)))
(binding [dom/node content-container]
(dom/div
(dom/props {:class (if (= active-tab tab-id)
"block"
"hidden")})
(PlayerV4. video)))))))))))
#2024-08-0311:20Dustin Getzportals are broken in v2, they only work in trivial cases in v2 because it was never intended to work#2024-08-0311:22Dustin Getzin v3 i expect this will work#2024-08-0311:24Dustin Getzreading the snippet closely - not sure, what you’re doing seems tricky i will need to try#2024-08-0311:25xificurCIn v2 rebinding dom/node portal-style is UB (undefined behavior)#2024-08-0314:16stagmoosegocha, thanks~#2024-08-0302:56stagmoosequick demo showing what i am building:
i always want a music looper for practicing guitar playing,
i want to save different clips for different video sources (youtube, vimeo, soundcloud, etc), and can quick navigating to previous created clips and loop a section.
with electric, now my project support various video sources (using thirdparty js lib),
crud video & abloops
can update db real fast and kind of "multiplayer" but i don't think this is useful for personal instrument practicing haha.
really rough at this moment, but can achieve some key features that i want.
thanks for clojure electric!#2024-08-0311:28Dustin Getzhow much effort to build? how many LOC?#2024-08-0314:15stagmoosearound 20~30hr i guess
and LOC is ~450
considering i only have really amateur four year clojure exp and not familiar with electric & datalevin before, i guess this is pretty good.#2024-08-0316:02Vincentawesome#2024-08-0316:03Vincentgreat work#2024-08-0412:18Dustin GetzSo, I just realized the video has sound. Sound on y'all! (Starting at about 0m20s)#2024-08-0412:28Dustin GetzI want to tweet this, but it needs to be edited down to something that folks can quickly understand what is happening. Can you re-record a 20-25 second version, where you set up two loops, and the loops sound nice, and play the loops for 10 seconds, so it is more clear what is happening?#2024-08-0412:39stagmoosefor sure.
i have two features to add before re-recording:
1. create a b points by cur player time
2. can adjust playing speed by adding buttons (like 0.75x 1x 1.5x ... buttons)#2024-08-0412:40Dustin Getzok#2024-08-0412:46Dustin Getza quick tip for a good twitter demo - minimize clicks and interactions, you get about 1 interaction per 3 seconds. This 15 second clip has 3 clicks and 2 words typed. Each click comes with exactly 1 straight line mouse movement, and the mouse movements are very slow, intentional, deliberate. Perhaps here there is only budget for setting up 1 loop and playing it (as long as it is a cool sounding loop)#2024-08-0312:15Olav FosseFrom what I understand, the only purpose of wrapping in (m/reductions …) is to emit the init value, correct?
(cc/defn debounce-discreet
([delay flow] (debounce-discreet delay nil flow))
;; Q: is m/reductions used only for emitting init
([delay init flow] (m/reductions {} init (m/ap (let [x (m/?< flow)]
(try (m/? (m/sleep delay x))
(catch Cancelled _ (m/amb))))))) )
#2024-08-0312:17Dustin Getzthat's right#2024-08-0312:17Olav FosseAwesome#2024-08-0312:18Olav FosseMissionary is starting to click for me which took a really long time 😅#2024-08-0312:18Dustin Getzfor others - electric's continuous time model (signals not streams) requires an initial value to be defined, so that for example the dom view is defined even when the first event has not arrived#2024-08-0313:43Olav FosseWhat learnings should I take away from the attached picture? Right now I’m confused about when I can or cannot mix and match Electric and Missionary code.#2024-08-0313:49Olav Fosse#2024-08-0314:35Dustin Getz:thinking_face:#2024-08-0314:36Dustin Getzsnippet 2 (commented out) is illegal because e/fn is not valid inside m/ap - as you figured out#2024-08-0314:42Dustin Getztry extracting the m/ap expression into a clojure function so electric doesn't see it - i think m/ap macroexpansion is incompatible with electric - but i was unable to find a ticket for it and i have forgotten#2024-08-0314:42Dustin Getziirc this is a very old bug that we haven't gotten around to fixing yet#2024-08-0315:08Dustin GetzRight now I’m confused about when I can or cannot mix and match Electric and Missionary codeClear answer = the cloroutine macros (m/ap, m/cp, m/sp) are not compatible with the Electric compiler, all other missionary operators are compatible. Workaround is to extract them to Clojure/Script functions and call those wrapper functions from Electric#2024-08-0401:33stagmooseon this page: https://electric.hyperfiddle.net/(electric-tutorial.tutorial-backpressure!%42ackpressure)/
it says if a tab is not focused, electric will not running?
is there a option to enable/disable this feature?
my player looping will break if a tab is not focused haha.#2024-08-0401:34Dustin Getzwhat will break?#2024-08-0401:35stagmoosei have a 500ms continous checker to see whether a player playing status is inside a ab-loop interval
if the tab is not focused, it will not do this continous checking#2024-08-0401:41Dustin GetzXY problem - can you tell me what problem you’re trying to solve using business words not peogramming words#2024-08-0401:47stagmoosei have a loop button and i want to make sure the player status is always inside so guitar players can practice for a loop (even if the tab is not focused)
on business side, do you mean user would not use it if the tab is not focused. so this checking not running is fine?#2024-08-0401:52stagmoosewow, i don't know the term "xy problem" before. just look it up, a quite helpful mindset i think.#2024-08-0409:57Dustin Getzthe tab is playing audio? and you want audio to continue when tab is background?#2024-08-0410:00Dustin Getzoh the javascript code to rewind the audio needs to run while tab is in the background#2024-08-0410:02Dustin Getzelectric doesn’t turn off javascript when the tab loses visibility, it just stops updating the dom. I think browser timers will continue to fire (subject to how browsers work - they may throttle timers)#2024-08-0410:04Dustin Getzi think this part of your logic is implemented with clojurescript and browser apis not electric right?#2024-08-0411:24stagmoosethis is what part of my code look like:#2024-08-0411:24stagmoosei use electric timer to trigger checking, if outside then rewind#2024-08-0411:25stagmooseso is it fine to use js/setInterval with electric?#2024-08-0411:26stagmoosealso, is there a built-in way in electric to know a tab is not focused?
my current status is the player will keep playing but not rewind.
so even i would go for the "pausing when not focusing" way, i should know the "not focused" event haha.#2024-08-0411:48Dustin Getzelectric-dom2 exposes https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_dom2.cljc#L269#2024-08-0411:48Dustin Getzactually we also expose it in electric.cljc - https://github.com/hyperfiddle/electric/blob/7e3400f005e0f62bf0c434dedc1fdd4c73f213ea/src/hyperfiddle/electric.cljc#L219-L235 (and use it to disable server clocks when not visible, otherwise your network will saturate with clock ticks especially mobile device radio)#2024-08-0411:51Dustin GetzYes, all browser apis are allowed in electric including setInterval etc#2024-08-0411:52Dustin GetzYou may run into additional problems with how browsers fire events when the tab is not visible, but at least they will not be electric problems#2024-08-0412:04stagmoosegotcha:ok_hand: thanks for your help! I would try them out.#2024-08-0402:10grounded_sageWhere would I look in the code to snip the backend expectations of the front end build on v2?#2024-08-0408:12xificurCYou mean run electric client-only? Our test setup includes a way, see l/defn and l/single+ here https://github.com/hyperfiddle/electric/blob/7e3400f005e0f62bf0c434dedc1fdd4c73f213ea/test/hyperfiddle/electric_test.cljc#L1888 . Note that this is not official#2024-08-0408:41grounded_sageUsing react on the front end < not official electric build ⚡#2024-08-0512:44Dustin GetzI am looking for 1-2 volunteers to be interviewed – someone who is Electric-skeptical (i.e. NOT already a user) – I would like to talk to you and write down your questions and concerns so that I can address them in upcoming content. DM me#2024-08-0613:11braai engineerknow a smart Elixir guy I've been trying to sell on Electric. Have reached out to him. Don't think he's used Electric yet#2024-08-0700:33Stef CoetzeeJS/TS interop might be good to include in content marketing. "Gateway drug" to Electric and Clojure more generally.#2024-08-0514:54Dustin GetzSeamless websocket reconnect in v3#2024-08-0515:03Dustin GetzActually I'm not sure this is demoing what I think it's demoing, I need a more violent test#2024-08-0515:11Dave MaysI was curious if the way electric's DOM works is a benefit or detriment to SEO for search engines that don't pre-parse JS? Maybe the question can better be framed as, is there a syntax or patter to serve SSR rendered HTML to the front end? Maybe this is already the default? (In Astro for example you can specify if the code that is generating your HTML should be run on the server or client).
Not saying it needs to work this way either, just getting a feel for how it might look.#2024-08-0515:17Dustin GetzSSR is not supported and not on the roadmap, sorry#2024-08-0515:18Dustin GetzElectric is for interactive apps, for dynamic websites there are many very very good options in javascript ecosystem#2024-08-0515:20henrikI imagine it might be possible to make an SEO friendly alternate world in e.g. pure backend Hiccup etc. Something that lays out the general content of a route without involving JS.#2024-08-0515:23henrikI.e., throw enough time/money at the problem, you can probably get SEO properties around, rather than in, Electric.#2024-08-0517:53Paulo CasarettoHi there! Currently exploring Eletric and thought a One Million Checkboxes kind of thing would be an interesting project to test drive.
I threw together the most stupid thing I could just to get started, and now I'm trying to make sense of whats actually going up and down the wire.
Looking at the websocket messages, I can see a pattern but I would like to understand the details. What are the different messages?
Also, there is one that is clearly transporting the state and I can see it is increasing in size. Why is that?
(ns electric-starter-app.main
(:require [hyperfiddle.electric :as e]
[hyperfiddle.electric-dom2 :as dom]
[hyperfiddle.electric-ui4 :as ui]))
;; Saving this file will automatically recompile and update in your browser
(def initial-state
(vec (repeat 1e3 false)))
#?(:clj (defonce !bools (atom initial-state))) ; server state
(e/def bools (e/server (e/watch !bools))) ; reactive signal derived from atom
(e/defn Main [ring-request]
(e/client
(binding [dom/node js/document.body]
(e/server
(e/for-by first [x (map-indexed vector bools)]
(e/client
(let [idx (e/server (first x))
item (e/server (last x))]
(ui/checkbox item (e/fn [v] (e/server (swap! !bools #(assoc % idx v))))))))))))#2024-08-0517:56Dustin Getze/for-by in v2 is a very bad operator. This for-by expression is (i think) collecting a list of nils and sending them back up to the server, in case the server needs to look at the collection returned by the for-by#2024-08-0517:56Dustin Getz(this is basically the central problem motivating the design of Electric v3)#2024-08-0517:59Dustin Getzas to wire traffic, there are two basic streams of information that are interleaved -
1. your program's reactive lexical scope
2. the evolving topology of your program's DAG (e.g. the outcome of any if statements (left or right), or the multiplicity of an e/for-by, stuff like that)#2024-08-0517:59Dustin GetzI also expect your app is very slow due to the O(n) nature of e/for-by#2024-08-0518:05Paulo CasarettoThan kind of makes sense, but with my current knowledge I'm feeling lost.
This is probably a symptom of the learning curve as I cling to past knowledge. If I were to design this app with an API, I would have a pretty good feeling for the kind of traffic (and therefore costs) my app would end up with.
How do you reach that with Eletric?#2024-08-0518:05Paulo Casaretto> I also expect your app is very slow due to the O(n) nature of e/for-by
Not really. It's pretty snappy. But I can't go over 10^4 checkboxes or the server crashes.#2024-08-0518:06Dustin Getzi'm surprised it crashes - but v2 is EOL at this point so we'll stress test in v3#2024-08-0518:07Paulo Casaretto#2024-08-0518:07Dustin Getznice#2024-08-0518:10Dustin Getzthe short answer is - even though electric v2 has made no attempt at all to produce optimized network traffic, it's still faster and cheaper in practice because API-style apps tend to build up big fat JSON documents and send them over the wire and then deserialize and unpack them on the other side at which point they get normalized or whatever to try to optimize the rendering before react goes and diffs the huge vdom changeset#2024-08-0518:11Dustin Getzthat's pretty much all waste#2024-08-0518:11Dustin Getzsending same JSON documents over the wire over and over again each time the user refreshes#2024-08-0518:12Dustin Getzi dont think anyone has actually measured, you rip out your reframe app with 7 second page load times or whatever and replace it with electric and you immediately have subsecond page loads, you're happy and we all leave it at that for now#2024-08-0518:13Dustin GetzNot the answer you're looking for, i know#2024-08-0518:15Paulo CasarettoYup, I believe in the principles you are building on. I think it's a better model than the one that's pervasive right now.#2024-08-0518:20Paulo CasarettoBut going back to my example, I'm at a loss as to how my changes will affect the traffic. For example:
1. how is the traffic going to be affected when I change from a vec of booleans to something closer to a bitfield
2. what does the (e/for-by first [x (map-indexed vector bools)] mean for traffic? should I be doing that on the client?#2024-08-0518:24Dustin Getzthe for-by implies diffing. since you correctly placed the for-by on the server, the input collection does not transfer, only the individual x values transfer (in your case (first x) and (last x)), which means as the bools reactively update, only changed x will subsequently transfer#2024-08-0518:25Dustin Getzthe result of the for-by getting streamed back to the server (`[nil nil nil ...]` ) is unavoidable in v2 - v3 paves all this#2024-08-0518:26Dustin Getzv3 formalizes the diffing in a more rigorous way and upgrades a bunch of stuff, tomorrow's blog post says more ( the blog post was previewed last week in this channel)#2024-08-0518:29Dustin Getzthere's also a bunch of nasty request/response stuff - which we haven't bothered to deal with yet because even with a bunch of IO waterfalls, electric is still faster than alternatives#2024-08-0518:30Paulo CasarettoAm I correct to assume then that (on v3) that will mean an initial load of the entire state and from then on just partial state updates?#2024-08-0518:30Dustin Getzthe model conceptually flattens out all the waterfalls - its just a matter of ; todo implement static lookahead#2024-08-0518:30Dustin Getzv3 is differential, you'll get a big squashed diff containing your zero to one state changeset#2024-08-0518:31Dustin Getz(or a stream of such diffs for each point of lexical scope)#2024-08-0518:32Dustin Getzprobably best to wait until it's out to discuss further#2024-08-0518:33Paulo CasarettoDeal#2024-08-0518:34Paulo CasarettoGoing to read the linked documentation again to get a deeper understanding#2024-08-0521:11Dustin Getz#2024-08-0521:46henrikIf the console is logging what’s going on the wire, that’s very compact#2024-08-0521:49telekidThis is one of the most mind-blowing demos I've seen in a long time.#2024-08-0521:49henrikHang on, this is all client scope, right?#2024-08-0521:57Dustin Getzit’s client only, diffs intercepted ~L20~ L26#2024-08-0521:59Dustin Getzin the networked case the traffic is a bit more involved than this but yes proportional, not linear#2024-08-0613:32Absolute NegativityWhat is a good way to channel a flow on the client to server? Here is my current ceremony:
(e/client
(let [x (new client-x>)]
;; might not get all x since discret flow becomes continuous
(e/server
(let [!send-fn (atom (fn [_]))
x> (m/observe (fn [!]
(reset! !send-fn !)
(fn [])))
send! (e/watch !send-fn)]
(send! x)
;; do thing with x>
))))
#2024-08-0613:49Absolute NegativityAlso using e/flow, I'm getting hyperfiddle.electric.Failure from the flow:
(e/client
(let [x (new client-x>)]
(e/server
(let [x> (e/flow x)]
;; do thing with x>
))))#2024-08-0613:53xificurCIIUC you have a missionary flow on the client and you want a missionary flow on the server? What's the initial problem? (XY problem)
What transfers are values, and you can lift them back to a missionary flow with e/fn
(e/client
(let [x (new client-x>)]
(e/server
(let [x> (e/fn [] x)]
..))))#2024-08-0615:13Absolute NegativityOh! I suppose x threw Pending to my handler and I didn't handle it. This fixed it:
(e/client
(let [bytes (new (scan> serial-port))]
(e/server
(let [tag-ids>
(read-epc>
(e/fn []
(try bytes
(catch hyperfiddle.electric.Pending _))))]
;; ...
))))
Am working on reading RFID tags from a bluetooth reader connected via web serial api. Thank you for keeping me sane.#2024-08-0615:35Dustin Getzyou want to stream a discrete flow over network and have it remain a discrete flow on the other side?#2024-08-0615:36Absolute NegativityYes. About to ask.#2024-08-0615:46Dustin GetzMoving a discrete flow over network without damaging it is a super-advanced use case, because Electric v2's internal operators are all continuous and essentially sprinkle m/latest at various points in the AST. The consequence is that if a m/latest touches your stream, values can and will be skipped, damaging your discrete flow.#2024-08-0615:51Dustin GetzThere are workarounds, such as reducing the events into a queue and moving the queue value across (using only safe electric syntax/operators that don't compile down to m/latest) and then unfolding the queue value back into events, but the problem likely poisons the workarounds as well#2024-08-0616:07Absolute NegativityLooks like I'm getting good enough data from the client even though it's not discrete. I had some bug preventing me from seeing new scan events (some binding for a tag id didn't change value). Thanks for the explanation, helps me understand Electric more.#2024-08-0618:46Olav FosseHow can I handle keyboard input? I just need to handle arrow keys :^)#2024-08-0618:51xificurCkeydown or keyup events#2024-08-0620:57Olav FosseCould you give an example? I’m familiar with JS events API 😊#2024-08-0622:31Vincent(dom/on "keyup" ...) right?#2024-08-0707:04xificurCyes. Have a look through the tutorials, they usually have examples of these things. E.g. https://electric.hyperfiddle.net/(electric-tutorial.demo-chat!%43hat)/ has a keydown handler#2024-08-0710:07Dustin Getzalso dom/on! - probably preferred over dom/on at this point#2024-08-0816:06Vincentthe exclamation point for the little sound the key makes when you tap it? 😅#2024-08-0816:13Dustin Getzwe use ! to generally mean 'imperative object or api'#2024-08-0710:17bruno.bonacciThe London Clojurians are happy to present:
*Electric Clojure v3: Differential Dataflow for UI (by Dustin Getz)*
https://www.meetup.com/London-Clojurians/events/302685725/#2024-08-0714:00danielneal@dustingetz just trying to read your article https://hyperfiddle-docs.notion.site/Electric-Clojure-v3-teaser-improved-transfer-semantics-2024-735b10c3a0dc424e93e060a0a3e80226 but got stuck at the example as I don’t know what e/cursor means. Would you mind elaborating a little?#2024-08-0714:24xificurCyou can think of it as looping over an electric collection. e/for-by in v2 does 2 things, diffing and looping, e/cursor is the second part only, diffing is done separately#2024-08-0714:35danielnealah nice, thanks#2024-08-0714:36xificurCfeel free to leave more feedback#2024-08-0714:37Dustin Getzstay tuned for next blog post!#2024-08-0714:37Dustin Getzalso the talk#2024-08-0714:37danielnealYes looking forward to that 🙂#2024-08-0714:41henrikDecomplect 💪 #2024-08-0714:46danielnealABD#2024-08-0714:46danielnealAlways Be Decomplecting#2024-08-0816:08Vincente/iterate-over-each-one#2024-08-0816:09Vincente/iterate-the-difference#2024-08-0715:21stagmoosenot directly clojure electric related.
the continuation of my previous looper project, i try to make it look nicer.
amazed at how good claude sonnet is! just screenshot old crappy web design photo and paste corresponding electric clojure code and copy the new tailwind generated class.
just crazily good!
"after / before" of the ab-loop slider#2024-08-0716:01Dustin Getzwow#2024-08-0716:01Dustin Getzclaude sonnet rewrote the css classes and any markup changes directly in electric and gave you the updated electric program as output?#2024-08-0718:52Jordan CalderwoodI have also used Claude to style my electric apps. Works really well. #2024-08-0723:15stagmooseclaude sonnet rewrote the css classes and any markup changes directly in electric and gave you the updated electric program as output?yeah, but i don't copy its all clojure electric code output
i just copy css class by hand and replace my old code's class because sometimes parenthesis balancing is not good haha.#2024-08-0807:07bdbrodieClaude converted a 900 LoC reagent ui to electric (v3) for me without trouble. Cursor does a good job with parens matching.#2024-08-0808:32henrikThat’s really impressive.#2024-08-0808:33henrik@U02KSQWSHSQ What LoC did you end up with? Did it write sensible Electric, or did you have tweak it? Was it correct on the first try?#2024-08-0811:02bdbrodieAt 820 LoC now. Had to tweak the e/for loops by hand, which I changed to e/for-by identity … as a naive first pass. Looks like I just want the enumeration part of e/for-by, so I’ll be testing e/cursor soon.#2024-08-0812:22Dustin GetzI am confused, electric v3 is not yet released#2024-08-0812:23Dustin Getzand it is also undocumented but for a blog post and a couple tiny examples, it is impossible for claude to understand electric v3 semantics#2024-08-0812:27bdbrodieindeed, it is only doing in context learning with the updated examples electric_starter_app#2024-08-0910:10henrikRPC is hot again https://react.dev/blog/2024/04/25/react-19#server-actions#2024-08-0912:00Dustin Getz> A common misunderstanding is that Server Components are denoted by "use server", but there is no directive for Server Components. The "use server" directive is used for Server Actions.
lol#2024-08-1012:56Olav FosseHow do I learn Electric? Rn I’m just sort of hacking things together and looking at examples and source code when I can’t get things to work. Is there a better way?#2024-08-1013:00Dustin Getzv2 was taught and learned by slack Q&A over an 18 month period. You kinda had to be there to participate in the discussions. Search is helpful to go back and read them. I know it's a shitty answer but we are financially constrained, all that JS ecosystem documentation is funded by a big venture capital bonfire#2024-08-1013:00Dustin Getzv3 is so much better, we are hoping problems like "cant get things to work" mostly disappear because the v3 semantics are so much more intuitive and all the big edge cases (needing explanation) are gone#2024-08-1013:02Dustin GetzIn principle, we'll now have more time to invest in examples and docs once v3 is launched. v3 development has soaked up 110% of our engineering bandwidth for 18 months now but that should taper off this year#2024-08-1013:03Dustin Getz(Commercially valuable projects can hire us of course)#2024-08-1013:31Olav FosseGotcha, I’ll make sure to catch the presentation on the 20.#2024-08-1013:49Tommy Jollyboat+1 for starting with v3; I suspect I'm still doing a few weird things to hack around issues that were resolved 6 or 12 months ago, and v3 will be a good time to try to un-learn those habits#2024-08-1013:51Tommy JollyboatAside from that I assume you found https://electric.hyperfiddle.net/ Electric Tutorial. It's terse, but good, and covers all the basic concepts of v2#2024-08-1020:56VincentThere are a collection of demos, in short you ought: determine which demos have the functionality required by your app and; smash them together into a frankenstein, critically ensuring everything still works every step along the way. The only way to electric is to start with a working copy and ensure it is working with every major change. @U027R2XBQEP